1
|
Castro-Pearson S, Samorodnitsky S, Yang K, Lotfi-Emran S, Ingraham NE, Bramante C, Jones EK, Greising S, Yu M, Steffen BT, Svensson J, Åhlberg E, Österberg B, Wacker D, Guan W, Puskarich M, Smed-Sörensen A, Lusczek E, Safo SE, Tignanelli CJ. Development of a proteomic signature associated with severe disease for patients with COVID-19 using data from 5 multicenter, randomized, controlled, and prospective studies. Sci Rep 2023; 13:20315. [PMID: 37985892 PMCID: PMC10661735 DOI: 10.1038/s41598-023-46343-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Accepted: 10/31/2023] [Indexed: 11/22/2023] Open
Abstract
Significant progress has been made in preventing severe COVID-19 disease through the development of vaccines. However, we still lack a validated baseline predictive biologic signature for the development of more severe disease in both outpatients and inpatients infected with SARS-CoV-2. The objective of this study was to develop and externally validate, via 5 international outpatient and inpatient trials and/or prospective cohort studies, a novel baseline proteomic signature, which predicts the development of moderate or severe (vs mild) disease in patients with COVID-19 from a proteomic analysis of 7000 + proteins. The secondary objective was exploratory, to identify (1) individual baseline protein levels and/or (2) protein level changes within the first 2 weeks of acute infection that are associated with the development of moderate/severe (vs mild) disease. For model development, samples collected from 2 randomized controlled trials were used. Plasma was isolated and the SomaLogic SomaScan platform was used to characterize protein levels for 7301 proteins of interest for all studies. We dichotomized 113 patients as having mild or moderate/severe COVID-19 disease. An elastic net approach was used to develop a predictive proteomic signature. For validation, we applied our signature to data from three independent prospective biomarker studies. We found 4110 proteins measured at baseline that significantly differed between patients with mild COVID-19 and those with moderate/severe COVID-19 after adjusting for multiple hypothesis testing. Baseline protein expression was associated with predicted disease severity with an error rate of 4.7% (AUC = 0.964). We also found that five proteins (Afamin, I-309, NKG2A, PRS57, LIPK) and patient age serve as a signature that separates patients with mild COVID-19 and patients with moderate/severe COVID-19 with an error rate of 1.77% (AUC = 0.9804). This panel was validated using data from 3 external studies with AUCs of 0.764 (Harvard University), 0.696 (University of Colorado), and 0.893 (Karolinska Institutet). In this study we developed and externally validated a baseline COVID-19 proteomic signature associated with disease severity for potential use in both outpatients and inpatients with COVID-19.
Collapse
Affiliation(s)
- Sandra Castro-Pearson
- Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN, USA
| | - Sarah Samorodnitsky
- Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN, USA
| | - Kaifeng Yang
- Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN, USA
| | - Sahar Lotfi-Emran
- Department of Medicine, University of Minnesota, Minneapolis, MN, USA
| | | | - Carolyn Bramante
- Department of Medicine, University of Minnesota, Minneapolis, MN, USA
| | - Emma K Jones
- Department of Surgery, University of Minnesota, 420 Delaware St SE, Minneapolis, MN, 55455, USA
| | - Sarah Greising
- School of Kinesiology, University of Minnesota, Minneapolis, MN, USA
| | - Meng Yu
- Division of Immunology and Allergy, Department of Medicine Solna, Center for Molecular Medicine, Karolinska Institutet and Karolinska University Hospital, Stockholm, Sweden
| | - Brian T Steffen
- Department of Surgery, University of Minnesota, 420 Delaware St SE, Minneapolis, MN, 55455, USA
| | - Julia Svensson
- Division of Immunology and Allergy, Department of Medicine Solna, Center for Molecular Medicine, Karolinska Institutet and Karolinska University Hospital, Stockholm, Sweden
| | - Eric Åhlberg
- Division of Immunology and Allergy, Department of Medicine Solna, Center for Molecular Medicine, Karolinska Institutet and Karolinska University Hospital, Stockholm, Sweden
| | - Björn Österberg
- Division of Immunology and Allergy, Department of Medicine Solna, Center for Molecular Medicine, Karolinska Institutet and Karolinska University Hospital, Stockholm, Sweden
| | - David Wacker
- Department of Medicine, University of Minnesota, Minneapolis, MN, USA
| | - Weihua Guan
- Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN, USA
| | - Michael Puskarich
- Department of Emergency Medicine, University of Minnesota, Minneapolis, MN, USA
- Department of Emergency Medicine, Hennepin County Medical Center, Minneapolis, MN, USA
| | - Anna Smed-Sörensen
- Division of Immunology and Allergy, Department of Medicine Solna, Center for Molecular Medicine, Karolinska Institutet and Karolinska University Hospital, Stockholm, Sweden
| | - Elizabeth Lusczek
- Department of Surgery, University of Minnesota, 420 Delaware St SE, Minneapolis, MN, 55455, USA
| | - Sandra E Safo
- Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN, USA
| | - Christopher J Tignanelli
- Department of Surgery, University of Minnesota, 420 Delaware St SE, Minneapolis, MN, 55455, USA.
- Institute for Health Informatics, University of Minnesota, Minneapolis, MN, USA.
| |
Collapse
|
3
|
Buttia C, Llanaj E, Raeisi-Dehkordi H, Kastrati L, Amiri M, Meçani R, Taneri PE, Ochoa SAG, Raguindin PF, Wehrli F, Khatami F, Espínola OP, Rojas LZ, de Mortanges AP, Macharia-Nimietz EF, Alijla F, Minder B, Leichtle AB, Lüthi N, Ehrhard S, Que YA, Fernandes LK, Hautz W, Muka T. Prognostic models in COVID-19 infection that predict severity: a systematic review. Eur J Epidemiol 2023; 38:355-372. [PMID: 36840867 PMCID: PMC9958330 DOI: 10.1007/s10654-023-00973-x] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Accepted: 01/28/2023] [Indexed: 02/26/2023]
Abstract
Current evidence on COVID-19 prognostic models is inconsistent and clinical applicability remains controversial. We performed a systematic review to summarize and critically appraise the available studies that have developed, assessed and/or validated prognostic models of COVID-19 predicting health outcomes. We searched six bibliographic databases to identify published articles that investigated univariable and multivariable prognostic models predicting adverse outcomes in adult COVID-19 patients, including intensive care unit (ICU) admission, intubation, high-flow nasal therapy (HFNT), extracorporeal membrane oxygenation (ECMO) and mortality. We identified and assessed 314 eligible articles from more than 40 countries, with 152 of these studies presenting mortality, 66 progression to severe or critical illness, 35 mortality and ICU admission combined, 17 ICU admission only, while the remaining 44 studies reported prediction models for mechanical ventilation (MV) or a combination of multiple outcomes. The sample size of included studies varied from 11 to 7,704,171 participants, with a mean age ranging from 18 to 93 years. There were 353 prognostic models investigated, with area under the curve (AUC) ranging from 0.44 to 0.99. A great proportion of studies (61.5%, 193 out of 314) performed internal or external validation or replication. In 312 (99.4%) studies, prognostic models were reported to be at high risk of bias due to uncertainties and challenges surrounding methodological rigor, sampling, handling of missing data, failure to deal with overfitting and heterogeneous definitions of COVID-19 and severity outcomes. While several clinical prognostic models for COVID-19 have been described in the literature, they are limited in generalizability and/or applicability due to deficiencies in addressing fundamental statistical and methodological concerns. Future large, multi-centric and well-designed prognostic prospective studies are needed to clarify remaining uncertainties.
Collapse
Affiliation(s)
- Chepkoech Buttia
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Emergency Department, Inselspital, Bern University Hospital, University of Bern, Freiburgstrasse 16C, 3010 Bern, Switzerland
- Epistudia, Bern, Switzerland
| | - Erand Llanaj
- Department of Molecular Epidemiology, German Institute of Human Nutrition Potsdam-Rehbrücke, Nuthetal, Germany
- ELKH-DE Public Health Research Group of the Hungarian Academy of Sciences, Department of Public Health and Epidemiology, Faculty of Medicine, University of Debrecen, Debrecen, Hungary
- Epistudia, Bern, Switzerland
- German Center for Diabetes Research (DZD), München-Neuherberg, Germany
| | - Hamidreza Raeisi-Dehkordi
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Lum Kastrati
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Graduate School for Health Sciences, University of Bern, Bern, Switzerland
- Department of Diabetes, Endocrinology, Nutritional Medicine and Metabolism, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| | - Mojgan Amiri
- Department of Epidemiology, Erasmus MC University Medical Center, Rotterdam, The Netherlands
| | - Renald Meçani
- Department of Pediatrics, “Mother Teresa” University Hospital Center, Tirana, University of Medicine, Tirana, Albania
- Division of Endocrinology and Diabetology, Department of Internal Medicine, Medical University of Graz, Graz, Austria
| | - Petek Eylul Taneri
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- HRB-Trials Methodology Research Network College of Medicine, Nursing and Health Sciences University of Galway, Galway, Ireland
| | | | - Peter Francis Raguindin
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Swiss Paraplegic Research, Nottwil, Switzerland
- Faculty of Health Sciences, University of Lucerne, Lucerne, Switzerland
| | - Faina Wehrli
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| | - Farnaz Khatami
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Graduate School for Health Sciences, University of Bern, Bern, Switzerland
- Department of Community Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - Octavio Pano Espínola
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Department of Preventive Medicine and Public Health, University of Navarre, Pamplona, Spain
- Navarra Institute for Health Research, IdiSNA, Pamplona, Spain
| | - Lyda Z. Rojas
- Research Group and Development of Nursing Knowledge (GIDCEN-FCV), Research Center, Cardiovascular Foundation of Colombia, Floridablanca, Santander, Colombia
| | | | | | - Fadi Alijla
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Graduate School for Health Sciences, University of Bern, Bern, Switzerland
| | - Beatrice Minder
- Public Health and Primary Care Library, University Library of Bern, University of Bern, Bern, Switzerland
| | - Alexander B. Leichtle
- University Institute of Clinical Chemistry, Inselspital, Bern University Hospital, and Center for Artificial Intelligence in Medicine (CAIM), University of Bern, Bern, Switzerland
| | - Nora Lüthi
- Emergency Department, Inselspital, Bern University Hospital, University of Bern, Freiburgstrasse 16C, 3010 Bern, Switzerland
| | - Simone Ehrhard
- Emergency Department, Inselspital, Bern University Hospital, University of Bern, Freiburgstrasse 16C, 3010 Bern, Switzerland
| | - Yok-Ai Que
- Department of Intensive Care Medicine, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| | - Laurenz Kopp Fernandes
- Deutsches Herzzentrum Berlin (DHZB), Berlin, Germany
- Charité Universitätsmedizin Berlin, Berlin, Germany
| | - Wolf Hautz
- Emergency Department, Inselspital, Bern University Hospital, University of Bern, Freiburgstrasse 16C, 3010 Bern, Switzerland
| | - Taulant Muka
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Epistudia, Bern, Switzerland
| |
Collapse
|
4
|
de Paiva BBM, Pereira PD, de Andrade CMV, Gomes VMR, Souza-Silva MVR, Martins KPMP, Sales TLS, de Carvalho RLR, Pires MC, Ramos LEF, Silva RT, de Freitas Martins Vieira A, Nunes AGS, de Oliveira Jorge A, de Oliveira Maurílio A, Scotton ALBA, da Silva CTCA, Cimini CCR, Ponce D, Pereira EC, Manenti ERF, Rodrigues FD, Anschau F, Botoni FA, Bartolazzi F, Grizende GMS, Noal HC, Duani H, Gomes IM, Costa JHSM, di Sabatino Santos Guimarães J, Tupinambás JT, Rugolo JM, Batista JDL, de Alvarenga JC, Chatkin JM, Ruschel KB, Zandoná LB, Pinheiro LS, Menezes LSM, de Oliveira LMC, Kopittke L, Assis LA, Marques LM, Raposo MC, Floriani MA, Bicalho MAC, Nogueira MCA, de Oliveira NR, Ziegelmann PK, Paraiso PG, de Lima Martelli PJ, Senger R, Menezes RM, Francisco SC, Araújo SF, Kurtz T, Fereguetti TO, de Oliveira TC, Ribeiro YCNMB, Ramires YC, Lima MCPB, Carneiro M, Bezerra AFB, Schwarzbold AV, de Moura Costa AS, Farace BL, Silveira DV, de Almeida Cenci EP, Lucas FB, Aranha FG, Bastos GAN, Vietta GG, Nascimento GF, Vianna HR, Guimarães HC, de Morais JDP, Moreira LB, de Oliveira LS, de Deus Sousa L, de Souza Viana L, de Souza Cabral MA, Ferreira MAP, de Godoy MF, de Figueiredo MP, Guimarães-Junior MH, de Paula de Sordi MA, da Cunha Severino Sampaio N, Assaf PL, Lutkmeier R, Valacio RA, Finger RG, de Freitas R, Guimarães SMM, Oliveira TF, Diniz THO, Gonçalves MA, Marcolino MS. Potential and limitations of machine meta-learning (ensemble) methods for predicting COVID-19 mortality in a large inhospital Brazilian dataset. Sci Rep 2023; 13:3463. [PMID: 36859446 PMCID: PMC9975879 DOI: 10.1038/s41598-023-28579-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2021] [Accepted: 01/20/2023] [Indexed: 03/03/2023] Open
Abstract
The majority of early prediction scores and methods to predict COVID-19 mortality are bound by methodological flaws and technological limitations (e.g., the use of a single prediction model). Our aim is to provide a thorough comparative study that tackles those methodological issues, considering multiple techniques to build mortality prediction models, including modern machine learning (neural) algorithms and traditional statistical techniques, as well as meta-learning (ensemble) approaches. This study used a dataset from a multicenter cohort of 10,897 adult Brazilian COVID-19 patients, admitted from March/2020 to November/2021, including patients [median age 60 (interquartile range 48-71), 46% women]. We also proposed new original population-based meta-features that have not been devised in the literature. Stacking has shown to achieve the best results reported in the literature for the death prediction task, improving over previous state-of-the-art by more than 46% in Recall for predicting death, with AUROC 0.826 and MacroF1 of 65.4%. The newly proposed meta-features were highly discriminative of death, but fell short in producing large improvements in final prediction performance, demonstrating that we are possibly on the limits of the prediction capabilities that can be achieved with the current set of ML techniques and (meta-)features. Finally, we investigated how the trained models perform on different hospitals, showing that there are indeed large differences in classifier performance between different hospitals, further making the case that errors are produced by factors that cannot be modeled with the current predictors.
Collapse
Affiliation(s)
- Bruno Barbosa Miranda de Paiva
- grid.8430.f0000 0001 2181 4888Computer Science Department, Universidade Federal de Minas Gerais, Av. Presidente Antônio Carlos, 6627, Belo Horizonte, Brazil
| | - Polianna Delfino Pereira
- grid.8430.f0000 0001 2181 4888Universidade Federal de Minas Gerais, Av. Presidente Antônio Carlos, 6627, Belo Horizonte, Brazil ,Institute for Health Technology Assessment (IATS/ CNPq), R. Ramiro Barcelos, 2359, building 21, room 507, Porto Alegre, Brazil
| | - Claudio Moisés Valiense de Andrade
- grid.8430.f0000 0001 2181 4888Computer Science Department, Universidade Federal de Minas Gerais, Av. Presidente Antônio Carlos, 6627, Belo Horizonte, Brazil
| | - Virginia Mara Reis Gomes
- grid.8430.f0000 0001 2181 4888Medical School and University Hospital, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 190, room 246, Belo Horizonte, Brazil
| | - Maira Viana Rego Souza-Silva
- grid.8430.f0000 0001 2181 4888Medical School and University Hospital, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 190, room 246, Belo Horizonte, Brazil
| | - Karina Paula Medeiros Prado Martins
- grid.8430.f0000 0001 2181 4888Medical School and University Hospital, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 190, room 246, Belo Horizonte, Brazil
| | - Thaís Lorenna Souza Sales
- grid.428481.30000 0001 1516 3599Universidade Federal de São João del-Rei, R. Sebastião Gonçalves Coelho, 400, Divinópolis, Brazil
| | | | - Magda Carvalho Pires
- grid.8430.f0000 0001 2181 4888Department of Statistics, Universidade Federal de Minas Gerais, Av. Presidente Antônio Carlos, 6627, ICEx, room 4071, Belo Horizonte, Brazil
| | - Lucas Emanuel Ferreira Ramos
- grid.8430.f0000 0001 2181 4888Department of Statistics, Universidade Federal de Minas Gerais, Av. Presidente Antônio Carlos, 6627, ICEx, room 4071, Belo Horizonte, Brazil
| | - Rafael Tavares Silva
- grid.8430.f0000 0001 2181 4888Department of Statistics, Universidade Federal de Minas Gerais, Av. Presidente Antônio Carlos, 6627, ICEx, room 4071, Belo Horizonte, Brazil
| | | | | | | | | | | | | | | | - Daniela Ponce
- grid.410543.70000 0001 2188 478XFaculdade de Medicina de Botucatu-Universidade Estadual Paulista “Júlio de Mesquita Filho”, Av. Prof. Mário Rubens Guimarães Montenegro, s/n-UNESP-Campus de Botucatu, Botucatu, Brazil
| | | | | | - Fernanda d’Athayde Rodrigues
- grid.414449.80000 0001 0125 3761Hospital de Clínicas de Porto Alegre, R. Ramiro Barcelos, 2350, Porto Alegre, Brazil
| | - Fernando Anschau
- grid.414914.dHospital Nossa Senhora da Conceição and Hospital Cristo Redentor, Av. Francisco Trein, 326, Porto Alegre, Brazil
| | | | - Frederico Bartolazzi
- Hospital Santo Antônio, Pç. Dr. Márcio Carvalho Lopes Filho, 501, Curvelo, Brazil
| | - Genna Maira Santos Grizende
- grid.477816.b0000 0004 4692 337XHospital Santa Casa de Misericórdia de Belo Horizonte, Av. Francisco Sales, 1111, Belo Horizonte, Brazil
| | - Helena Carolina Noal
- grid.411239.c0000 0001 2284 6531Universidade Federal de Santa Maria/Hospital Universitário/EBSERH, Av. Roraima, 1000, building 22, Santa Maria, Brazil
| | - Helena Duani
- grid.8430.f0000 0001 2181 4888Medical School and University Hospital, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 190, room 246, Belo Horizonte, Brazil
| | - Isabela Moraes Gomes
- grid.8430.f0000 0001 2181 4888Medical School and University Hospital, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 190, room 246, Belo Horizonte, Brazil
| | | | | | | | - Juliana Machado Rugolo
- grid.410543.70000 0001 2188 478XFaculdade de Medicina de Botucatu-Universidade Estadual Paulista “Júlio de Mesquita Filho”, Av. Prof. Mário Rubens Guimarães Montenegro, s/n-UNESP-Campus de Botucatu, Botucatu, Brazil
| | - Joanna d’Arc Lyra Batista
- grid.440565.60000 0004 0491 0431Universidade Federal da Fronteira Sul, Av. Fernando Machado, 108E, Chapecó, Brazil
| | | | - José Miguel Chatkin
- grid.411379.90000 0001 2198 7041Hospital São Lucas PUCRS, Av. Ipiranga, 6690, Porto Alegre, Brazil
| | - Karen Brasil Ruschel
- grid.414871.f0000 0004 0491 7596Hospital Mãe de Deus, R. José de Alencar, 286, Porto Alegre, Brazil
| | | | | | - Luanna Silva Monteiro Menezes
- Hospital Metropolitano Odilon Behrens, R. Formiga, 50, Belo Horizonte, Brazil ,Hospital Luxemburgo, R. Gentios, 1350, Belo Horizonte, Brazil
| | | | - Luciane Kopittke
- grid.414914.dHospital Nossa Senhora da Conceição and Hospital Cristo Redentor, Av. Francisco Trein, 326, Porto Alegre, Brazil
| | - Luisa Argolo Assis
- grid.412520.00000 0001 2155 6671Pontifícia Universidade Católica de Minas Gerais, Av. Dom José Gaspar, 500, Belo Horizonte, Brazil
| | - Luiza Margoto Marques
- grid.419130.e0000 0004 0413 0953Faculdade de Ciências Médicas de Minas Gerais, Al. Ezequiel Dias, 275, Belo Horizonte, Brazil
| | - Magda Cesar Raposo
- grid.428481.30000 0001 1516 3599Universidade Federal de São João del-Rei, R. Sebastião Gonçalves Coelho, 400, Divinópolis, Brazil
| | - Maiara Anschau Floriani
- grid.414856.a0000 0004 0398 2134Hospital Moinhos de Vento, R. Ramiro Barcelos, 910, Porto Alegre, Brazil ,Moinhos Research Institute, 910 Ramiro Barcelos Street, 5 floor, Porto Alegre, Brazil
| | - Maria Aparecida Camargos Bicalho
- grid.452464.50000 0000 9270 1314Fundação Hospitalar do Estado de Minas Gerais–FHEMIG, Cidade Administrativa de Minas Gerais, Edifício Gerais, 13rd floor, Rod. Papa João Paulo II, 3777, Belo Horizonte, Brazil
| | | | - Neimy Ramos de Oliveira
- grid.452464.50000 0000 9270 1314Hospital Eduardo de Menezes, R. Dr. Cristiano Rezende, 2213, Belo Horizonte, Brazil
| | | | | | | | - Roberta Senger
- grid.411239.c0000 0001 2284 6531Universidade Federal de Santa Maria/Hospital Universitário/EBSERH, Av. Roraima, 1000, building 22, Santa Maria, Brazil
| | | | | | | | - Tatiana Kurtz
- Hospital Santa Cruz, R. Fernando Abott, 174, Santa Cruz do Sul, Brazil
| | - Tatiani Oliveira Fereguetti
- grid.452464.50000 0000 9270 1314Hospital Eduardo de Menezes, R. Dr. Cristiano Rezende, 2213, Belo Horizonte, Brazil
| | | | | | | | | | - Marcelo Carneiro
- Hospital Santa Cruz, R. Fernando Abott, 174, Santa Cruz do Sul, Brazil
| | | | - Alexandre Vargas Schwarzbold
- grid.411239.c0000 0001 2284 6531Universidade Federal de Santa Maria/Hospital Universitário/EBSERH, Av. Roraima, 1000, building 22, Santa Maria, Brazil
| | | | - Barbara Lopes Farace
- grid.490178.3Hospital Risoleta Tolentino Neves, R. das Gabirobas, 01, Belo Horizonte, Brazil
| | | | | | | | | | - Gisele Alsina Nader Bastos
- grid.414856.a0000 0004 0398 2134Hospital Moinhos de Vento, R. Ramiro Barcelos, 910, Porto Alegre, Brazil
| | | | | | | | | | | | - Leila Beltrami Moreira
- grid.414449.80000 0001 0125 3761Hospital de Clínicas de Porto Alegre, R. Ramiro Barcelos, 2350, Porto Alegre, Brazil
| | | | | | | | - Máderson Alvares de Souza Cabral
- grid.8430.f0000 0001 2181 4888Medical School and University Hospital, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 190, room 246, Belo Horizonte, Brazil
| | - Maria Angélica Pires Ferreira
- grid.414449.80000 0001 0125 3761Hospital de Clínicas de Porto Alegre, R. Ramiro Barcelos, 2350, Porto Alegre, Brazil
| | - Mariana Frizzo de Godoy
- grid.411379.90000 0001 2198 7041Hospital São Lucas PUCRS, Av. Ipiranga, 6690, Porto Alegre, Brazil
| | | | | | - Mônica Aparecida de Paula de Sordi
- grid.410543.70000 0001 2188 478XFaculdade de Medicina de Botucatu-Universidade Estadual Paulista “Júlio de Mesquita Filho”, Av. Prof. Mário Rubens Guimarães Montenegro, s/n-UNESP-Campus de Botucatu, Botucatu, Brazil
| | | | - Pedro Ledic Assaf
- Hospital Metropolitano Doutor Célio de Castro, R. Dona Luiza, 311, Belo Horizonte, Brazil
| | - Raquel Lutkmeier
- grid.414914.dHospital Nossa Senhora da Conceição and Hospital Cristo Redentor, Av. Francisco Trein, 326, Porto Alegre, Brazil
| | | | | | - Rufino de Freitas
- Hospital São João de Deus, R. do Cobre, 800, São João de Deus, Brazil
| | | | | | | | - Marcos André Gonçalves
- grid.8430.f0000 0001 2181 4888Computer Science Department, Universidade Federal de Minas Gerais, Av. Presidente Antônio Carlos, 6627, Belo Horizonte, Brazil
| | - Milena Soriano Marcolino
- Institute for Health Technology Assessment (IATS/ CNPq), R. Ramiro Barcelos, 2359, building 21, room 507, Porto Alegre, Brazil. .,Medical School and University Hospital, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 190, room 246, Belo Horizonte, Brazil. .,Telehealth Center, University Hospital, Universidade Federal de Minas Gerais, Avenida Professor Alfredo Balena, 110 room 107. Santa Efigênia, Belo Horizonte, MG, CEP 30130-100, Brazil.
| |
Collapse
|