201
|
Zhu Y, Yu B, Tang K, Liu T, Niu D, Zhang L. Development and validation of a prediction model based on comorbidities to estimate the risk of in-hospital death in patients with COVID-19. Front Public Health 2023; 11:1194349. [PMID: 37304114 PMCID: PMC10254410 DOI: 10.3389/fpubh.2023.1194349] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2023] [Accepted: 05/12/2023] [Indexed: 06/13/2023] Open
Abstract
Background Most existing prognostic models of COVID-19 require imaging manifestations and laboratory results as predictors, which are only available in the post-hospitalization period. Therefore, we aimed to develop and validate a prognostic model to assess the in-hospital death risk in COVID-19 patients using routinely available predictors at hospital admission. Methods We conducted a retrospective cohort study of patients with COVID-19 using the Healthcare Cost and Utilization Project State Inpatient Database in 2020. Patients hospitalized in Eastern United States (Florida, Michigan, Kentucky, and Maryland) were included in the training set, and those hospitalized in Western United States (Nevada) were included in the validation set. Discrimination, calibration, and clinical utility were evaluated to assess the model's performance. Results A total of 17 954 in-hospital deaths occurred in the training set (n = 168 137), and 1,352 in-hospital deaths occurred in the validation set (n = 12 577). The final prediction model included 15 variables readily available at hospital admission, including age, sex, and 13 comorbidities. This prediction model showed moderate discrimination with an area under the curve (AUC) of 0.726 (95% confidence interval [CI]: 0.722-0.729) and good calibration (Brier score = 0.090, slope = 1, intercept = 0) in the training set; a similar predictive ability was observed in the validation set. Conclusion An easy-to-use prognostic model based on predictors readily available at hospital admission was developed and validated for the early identification of COVID-19 patients with a high risk of in-hospital death. This model can be a clinical decision-support tool to triage patients and optimize resource allocation.
Collapse
Affiliation(s)
- Yangjie Zhu
- Department of Military Health Management, College of Health Service, Naval Medical University, Shanghai, China
| | - Boyang Yu
- Department of Military Health Management, College of Health Service, Naval Medical University, Shanghai, China
- Department of Medical Health Service, General Hospital of Northern Theater Command of PLA, Shenyang, China
| | - Kang Tang
- Department of Military Health Management, College of Health Service, Naval Medical University, Shanghai, China
| | - Tongtong Liu
- Department of Military Health Management, College of Health Service, Naval Medical University, Shanghai, China
- Department of Medical Health Service, 969th Hospital of PLA Joint Logistics Support Forces, Hohhot, China
| | - Dongjun Niu
- Department of Military Health Management, College of Health Service, Naval Medical University, Shanghai, China
| | - Lulu Zhang
- Department of Military Health Management, College of Health Service, Naval Medical University, Shanghai, China
| |
Collapse
|
202
|
Rajpurkar P, Lungren MP. The Current and Future State of AI Interpretation of Medical Images. N Engl J Med 2023; 388:1981-1990. [PMID: 37224199 DOI: 10.1056/nejmra2301725] [Citation(s) in RCA: 97] [Impact Index Per Article: 97.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/26/2023]
Affiliation(s)
- Pranav Rajpurkar
- From the Department of Biomedical Informatics, Harvard Medical School, Boston (P.R.); the Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Stanford, and the Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco - both in California (M.P.L.); and Microsoft, Redmond, Washington (M.P.L.)
| | - Matthew P Lungren
- From the Department of Biomedical Informatics, Harvard Medical School, Boston (P.R.); the Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Stanford, and the Department of Radiology and Biomedical Imaging, University of California, San Francisco, San Francisco - both in California (M.P.L.); and Microsoft, Redmond, Washington (M.P.L.)
| |
Collapse
|
203
|
Kołodziejczak MM, Sierakowska K, Tkachenko Y, Kowalski P. Artificial Intelligence in the Intensive Care Unit: Present and Future in the COVID-19 Era. J Pers Med 2023; 13:891. [PMID: 37373880 PMCID: PMC10304011 DOI: 10.3390/jpm13060891] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2023] [Revised: 05/18/2023] [Accepted: 05/22/2023] [Indexed: 06/29/2023] Open
Abstract
The development of artificial intelligence (AI) allows for the construction of technologies capable of implementing functions that represent the human mind, senses, and problem-solving skills, leading to automation, rapid data analysis, and acceleration of tasks. These solutions has been initially implemented in medical fields relying on image analysis; however, technological development and interdisciplinary collaboration allows for the introduction of AI-based enhancements to further medical specialties. During the COVID-19 pandemic, novel technologies established on big data analysis experienced a rapid expansion. Yet, despite the possibilities of advancements with these AI technologies, there are number of shortcomings that need to be resolved to assert the highest and the safest level of performance, especially in the setting of the intensive care unit (ICU). Within the ICU, numerous factors and data affect clinical decision making and work management that could be managed by AI-based technologies. Early detection of a patient's deterioration, identification of unknown prognostic parameters, or even improvement of work organization are a few of many areas where patients and medical personnel can benefit from solutions developed with AI.
Collapse
Affiliation(s)
- Michalina Marta Kołodziejczak
- Department of Anesthesiology and Intensive Care, Collegium Medicum Bydgoszcz, Nicolaus Copernicus University Torun, Antoni Jurasz University Hospital No.1, 85-094 Bydgoszcz, Poland;
| | - Katarzyna Sierakowska
- Department of Anesthesiology and Intensive Care, Collegium Medicum Bydgoszcz, Nicolaus Copernicus University Torun, Antoni Jurasz University Hospital No.1, 85-094 Bydgoszcz, Poland;
| | - Yurii Tkachenko
- Department of Anesthesiology and Intensive Care, Władysław Biegański Regional Specialized Hospital, 86-300 Grudziadz, Poland; (Y.T.); (P.K.)
| | - Piotr Kowalski
- Department of Anesthesiology and Intensive Care, Władysław Biegański Regional Specialized Hospital, 86-300 Grudziadz, Poland; (Y.T.); (P.K.)
| |
Collapse
|
204
|
Salichos L, Warrell J, Cevasco H, Chung A, Gerstein M. Genetic determination of regional connectivity in modelling the spread of COVID-19 outbreak for more efficient mitigation strategies. Sci Rep 2023; 13:8470. [PMID: 37231011 DOI: 10.1038/s41598-023-34959-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Accepted: 05/10/2023] [Indexed: 05/27/2023] Open
Abstract
For the COVID-19 pandemic, viral transmission has been documented in many historical and geographical contexts. Nevertheless, few studies have explicitly modeled the spatiotemporal flow based on genetic sequences, to develop mitigation strategies. Additionally, thousands of SARS-CoV-2 genomes have been sequenced with associated records, potentially providing a rich source for such spatiotemporal analysis, an unprecedented amount during a single outbreak. Here, in a case study of seven states, we model the first wave of the outbreak by determining regional connectivity from phylogenetic sequence information (i.e. "genetic connectivity"), in addition to traditional epidemiologic and demographic parameters. Our study shows nearly all of the initial outbreak can be traced to a few lineages, rather than disconnected outbreaks, indicative of a mostly continuous initial viral flow. While the geographic distance from hotspots is initially important in the modeling, genetic connectivity becomes increasingly significant later in the first wave. Moreover, our model predicts that isolated local strategies (e.g. relying on herd immunity) can negatively impact neighboring regions, suggesting more efficient mitigation is possible with unified, cross-border interventions. Finally, our results suggest that a few targeted interventions based on connectivity can have an effect similar to that of an overall lockdown. They also suggest that while successful lockdowns are very effective in mitigating an outbreak, less disciplined lockdowns quickly decrease in effectiveness. Our study provides a framework for combining phylodynamic and computational methods to identify targeted interventions.
Collapse
Affiliation(s)
- Leonidas Salichos
- Program in Computational Biology and Bioinformatics, Yale University, New Haven, CT, 06520, USA.
- Biological and Chemical Sciences, New York Institute of Technology, Manhattan, NY, 10023, USA.
| | - Jonathan Warrell
- Program in Computational Biology and Bioinformatics, Yale University, New Haven, CT, 06520, USA
| | - Hannah Cevasco
- Program in Computational Biology and Bioinformatics, Yale University, New Haven, CT, 06520, USA
| | - Alvin Chung
- Program in Computational Biology and Bioinformatics, Yale University, New Haven, CT, 06520, USA
| | - Mark Gerstein
- Program in Computational Biology and Bioinformatics, Yale University, New Haven, CT, 06520, USA.
- Department of Molecular Biophysics and Biochemistry, Yale University, New Haven, CT, 06520, USA.
- Department of Computer Science, Yale University, New Haven, CT, 06520, USA.
- Center for Biomedical Data Science, Yale University, New Haven, CT, 06520, USA.
- Department of Statistics & Data Science, Yale University, New Haven, CT, 06520, USA.
| |
Collapse
|
205
|
Pavel I, Ciocoiu IB. COVID-19 Detection from Cough Recordings Using Bag-of-Words Classifiers. SENSORS (BASEL, SWITZERLAND) 2023; 23:4996. [PMID: 37299721 PMCID: PMC10255075 DOI: 10.3390/s23114996] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/20/2023] [Revised: 05/21/2023] [Accepted: 05/22/2023] [Indexed: 06/12/2023]
Abstract
Reliable detection of COVID-19 from cough recordings is evaluated using bag-of-words classifiers. The effect of using four distinct feature extraction procedures and four different encoding strategies is evaluated in terms of the Area Under Curve (AUC), accuracy, sensitivity, and F1-score. Additional studies include assessing the effect of both input and output fusion approaches and a comparative analysis against 2D solutions using Convolutional Neural Networks. Extensive experiments conducted on the COUGHVID and COVID-19 Sounds datasets indicate that sparse encoding yields the best performances, showing robustness against various combinations of feature type, encoding strategy, and codebook dimension parameters.
Collapse
Affiliation(s)
| | - Iulian B. Ciocoiu
- Faculty of Electronics, Telecommunications and Information Technology, “Gheorghe Asachi” Technical University of Iasi, Bd. Carol I 11A, 700050 Iasi, Romania;
| |
Collapse
|
206
|
Lyons J, Nafilyan V, Akbari A, Bedston S, Harrison E, Hayward A, Hippisley-Cox J, Kee F, Khunti K, Rahman S, Sheikh A, Torabi F, Lyons RA. An external validation of the QCOVID3 risk prediction algorithm for risk of hospitalisation and death from COVID-19: An observational, prospective cohort study of 1.66m vaccinated adults in Wales, UK. PLoS One 2023; 18:e0285979. [PMID: 37200350 PMCID: PMC10194890 DOI: 10.1371/journal.pone.0285979] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2023] [Accepted: 05/07/2023] [Indexed: 05/20/2023] Open
Abstract
INTRODUCTION At the start of the COVID-19 pandemic there was an urgent need to identify individuals at highest risk of severe outcomes, such as hospitalisation and death following infection. The QCOVID risk prediction algorithms emerged as key tools in facilitating this which were further developed during the second wave of the COVID-19 pandemic to identify groups of people at highest risk of severe COVID-19 related outcomes following one or two doses of vaccine. OBJECTIVES To externally validate the QCOVID3 algorithm based on primary and secondary care records for Wales, UK. METHODS We conducted an observational, prospective cohort based on electronic health care records for 1.66m vaccinated adults living in Wales on 8th December 2020, with follow-up until 15th June 2021. Follow-up started from day 14 post vaccination to allow the full effect of the vaccine. RESULTS The scores produced by the QCOVID3 risk algorithm showed high levels of discrimination for both COVID-19 related deaths and hospital admissions and good calibration (Harrell C statistic: ≥ 0.828). CONCLUSION This validation of the updated QCOVID3 risk algorithms in the adult vaccinated Welsh population has shown that the algorithms are valid for use in the Welsh population, and applicable on a population independent of the original study, which has not been previously reported. This study provides further evidence that the QCOVID algorithms can help inform public health risk management on the ongoing surveillance and intervention to manage COVID-19 related risks.
Collapse
Affiliation(s)
- Jane Lyons
- Faculty of Medicine, Health & Life Science, Population Data Science, Swansea University Medical School, Swansea University, Swansea, United Kingdom
| | - Vahé Nafilyan
- Office of National Statistics, Newport, United Kingdom
| | - Ashley Akbari
- Faculty of Medicine, Health & Life Science, Population Data Science, Swansea University Medical School, Swansea University, Swansea, United Kingdom
| | - Stuart Bedston
- Faculty of Medicine, Health & Life Science, Population Data Science, Swansea University Medical School, Swansea University, Swansea, United Kingdom
| | - Ewen Harrison
- Usher Institute, Centre for Medical Informatics, University of Edinburgh, Edinburgh, United Kingdom
| | - Andrew Hayward
- Department of Epidemiology and Public Health, University College London, London, United Kingdom
| | - Julia Hippisley-Cox
- Nuffield Department, Primary Care Health Sciences, University of Oxford, Oxford, United Kingdom
| | - Frank Kee
- School of Medicine, Dentistry and Biomedical Sciences, Queen’s University Belfast, Belfast, United Kingdom
| | - Kamlesh Khunti
- Diabetes Research Centre, University of Leicester, Leicester, United Kingdom
| | - Shamim Rahman
- Department of Health and Social Care, Mental Health and Disabilities Analysis, London, United Kingdom
| | - Aziz Sheikh
- Usher Institute, University of Edinburgh, Edinburgh, United Kingdom
| | - Fatemeh Torabi
- Faculty of Medicine, Health & Life Science, Population Data Science, Swansea University Medical School, Swansea University, Swansea, United Kingdom
| | - Ronan A. Lyons
- Faculty of Medicine, Health & Life Science, Population Data Science, Swansea University Medical School, Swansea University, Swansea, United Kingdom
| |
Collapse
|
207
|
Pokhriyal N, Koebe T. AI-assisted diplomatic decision-making during crises-Challenges and opportunities. Front Big Data 2023; 6:1183313. [PMID: 37252128 PMCID: PMC10213620 DOI: 10.3389/fdata.2023.1183313] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2023] [Accepted: 04/21/2023] [Indexed: 05/31/2023] Open
Affiliation(s)
- Neeti Pokhriyal
- Department of Computer Science, Dartmouth College, Hanover, NH, United States
| | - Till Koebe
- Saarland Informatics Campus, Universität des Saarlandes, Saarbrücken, Germany
| |
Collapse
|
208
|
Fraser AG, Biasin E, Bijnens B, Bruining N, Caiani EG, Cobbaert K, Davies RH, Gilbert SH, Hovestadt L, Kamenjasevic E, Kwade Z, McGauran G, O'Connor G, Vasey B, Rademakers FE. Artificial intelligence in medical device software and high-risk medical devices - a review of definitions, expert recommendations and regulatory initiatives. Expert Rev Med Devices 2023; 20:467-491. [PMID: 37157833 DOI: 10.1080/17434440.2023.2184685] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
INTRODUCTION Artificial intelligence (AI) encompasses a wide range of algorithms with risks when used to support decisions about diagnosis or treatment, so professional and regulatory bodies are recommending how they should be managed. AREAS COVERED AI systems may qualify as standalone medical device software (MDSW) or be embedded within a medical device. Within the European Union (EU) AI software must undergo a conformity assessment procedure to be approved as a medical device. The draft EU Regulation on AI proposes rules that will apply across industry sectors, while for devices the Medical Device Regulation also applies. In the CORE-MD project (Coordinating Research and Evidence for Medical Devices), we have surveyed definitions and summarize initiatives made by professional consensus groups, regulators, and standardization bodies. EXPERT OPINION The level of clinical evidence required should be determined according to each application and to legal and methodological factors that contribute to risk, including accountability, transparency, and interpretability. EU guidance for MDSW based on international recommendations does not yet describe the clinical evidence needed for medical AI software. Regulators, notified bodies, manufacturers, clinicians and patients would all benefit from common standards for the clinical evaluation of high-risk AI applications and transparency of their evidence and performance.
Collapse
Affiliation(s)
- Alan G Fraser
- University Hospital of Wales, School of Medicine, Cardiff University, Heath Park, Cardiff, U.K
- KU Leuven, Leuven, Belgium
| | | | - Bart Bijnens
- Engineering Sciences, Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
| | - Nico Bruining
- Department of Clinical and Experimental Information processing (Digital Cardiology), Erasmus Medical Center, Thoraxcenter, Rotterdam, the Netherlands
| | - Enrico G Caiani
- Department of Electronics, Information and Biomedical Engineering, Politecnico di Milano, Milan, Italy
| | | | - Rhodri H Davies
- Institute of Cardiovascular Science, University College London, London, U.K
| | - Stephen H Gilbert
- Technische Universität Dresden, Else Kröner Fresenius Center for Digital Health, Dresden, Germany
| | | | | | | | | | | | - Baptiste Vasey
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK
| | | |
Collapse
|
209
|
de Hond AAH, Shah VB, Kant IMJ, Van Calster B, Steyerberg EW, Hernandez-Boussard T. Perspectives on validation of clinical predictive algorithms. NPJ Digit Med 2023; 6:86. [PMID: 37149704 PMCID: PMC10163568 DOI: 10.1038/s41746-023-00832-9] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2023] [Accepted: 04/28/2023] [Indexed: 05/08/2023] Open
Affiliation(s)
- Anne A H de Hond
- Clinical AI Implementation and Research Lab, Leiden University Medical Centre, Leiden, the Netherlands.
- Department of Medicine (Biomedical Informatics), Stanford University, Stanford, CA, USA.
- Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, the Netherlands.
| | - Vaibhavi B Shah
- Department of Medicine (Biomedical Informatics), Stanford University, Stanford, CA, USA
| | - Ilse M J Kant
- Department of Digital Health, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Ben Van Calster
- Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, the Netherlands
- Department of Development & Regeneration, KU Leuven, Leuven, Belgium
| | - Ewout W Steyerberg
- Clinical AI Implementation and Research Lab, Leiden University Medical Centre, Leiden, the Netherlands
- Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, the Netherlands
| | - Tina Hernandez-Boussard
- Department of Medicine (Biomedical Informatics), Stanford University, Stanford, CA, USA
- Department of Biomedical Data Science, Stanford University, Stanford, CA, USA
- Department of Epidemiology & Population Health (by courtesy), Stanford University, Stanford, CA, USA
| |
Collapse
|
210
|
Wu Y, Dravid A, Wehbe RM, Katsaggelos AK. DeepCOVID-Fuse: A Multi-Modality Deep Learning Model Fusing Chest X-rays and Clinical Variables to Predict COVID-19 Risk Levels. Bioengineering (Basel) 2023; 10:556. [PMID: 37237626 PMCID: PMC10215672 DOI: 10.3390/bioengineering10050556] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Revised: 04/28/2023] [Accepted: 05/02/2023] [Indexed: 05/28/2023] Open
Abstract
The COVID-19 pandemic has posed unprecedented challenges to global healthcare systems, highlighting the need for accurate and timely risk prediction models that can prioritize patient care and allocate resources effectively. This study presents DeepCOVID-Fuse, a deep learning fusion model that predicts risk levels in patients with confirmed COVID-19 by combining chest radiographs (CXRs) and clinical variables. The study collected initial CXRs, clinical variables, and outcomes (i.e., mortality, intubation, hospital length of stay, Intensive care units (ICU) admission) from February to April 2020, with risk levels determined by the outcomes. The fusion model was trained on 1657 patients (Age: 58.30 ± 17.74; Female: 807) and validated on 428 patients (56.41 ± 17.03; 190) from the local healthcare system and tested on 439 patients (56.51 ± 17.78; 205) from a different holdout hospital. The performance of well-trained fusion models on full or partial modalities was compared using DeLong and McNemar tests. Results show that DeepCOVID-Fuse significantly (p < 0.05) outperformed models trained only on CXRs or clinical variables, with an accuracy of 0.658 and an area under the receiver operating characteristic curve (AUC) of 0.842. The fusion model achieves good outcome predictions even when only one of the modalities is used in testing, demonstrating its ability to learn better feature representations across different modalities during training.
Collapse
Affiliation(s)
- Yunan Wu
- Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL 60201, USA;
| | - Amil Dravid
- Department of Computer Science, Northwestern University, Evanston, IL 60201, USA;
| | - Ramsey Michael Wehbe
- The Division of Cardiology, Department of Medicine and Bluhm Cardiovascular Institute, Northwestern Memorial Hospital, Chicago, IL 60611, USA;
| | - Aggelos K. Katsaggelos
- Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL 60201, USA;
- Department of Computer Science, Northwestern University, Evanston, IL 60201, USA;
| |
Collapse
|
211
|
Banoei MM, Rafiepoor H, Zendehdel K, Seyyedsalehi MS, Nahvijou A, Allameh F, Amanpour S. Unraveling complex relationships between COVID-19 risk factors using machine learning based models for predicting mortality of hospitalized patients and identification of high-risk group: a large retrospective study. Front Med (Lausanne) 2023; 10:1170331. [PMID: 37215714 PMCID: PMC10192907 DOI: 10.3389/fmed.2023.1170331] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2023] [Accepted: 04/11/2023] [Indexed: 05/24/2023] Open
Abstract
Background At the end of 2019, the coronavirus disease 2019 (COVID-19) pandemic increased the hospital burden of COVID-19 caused by the SARS-Cov-2 and became the most significant health challenge for nations worldwide. The severity and high mortality of COVID-19 have been correlated with various demographic characteristics and clinical manifestations. Prediction of mortality rate, identification of risk factors, and classification of patients played a crucial role in managing COVID-19 patients. Our purpose was to develop machine learning (ML)-based models for the prediction of mortality and severity among patients with COVID-19. Identifying the most important predictors and unraveling their relationships by classification of patients to the low-, moderate- and high-risk groups might guide prioritizing treatment decisions and a better understanding of interactions between factors. A detailed evaluation of patient data is believed to be important since COVID-19 resurgence is underway in many countries. Results The findings of this study revealed that the ML-based statistically inspired modification of the partial least square (SIMPLS) method could predict the in-hospital mortality among COVID-19 patients. The prediction model was developed using 19 predictors including clinical variables, comorbidities, and blood markers with moderate predictability (Q2 = 0.24) to separate survivors and non-survivors. Oxygen saturation level, loss of consciousness, and chronic kidney disease (CKD) were the top mortality predictors. Correlation analysis showed different correlation patterns among predictors for each non-survivor and survivor cohort separately. The main prediction model was verified using other ML-based analyses with a high area under the curve (AUC) (0.81-0.93) and specificity (0.94-0.99). The obtained data revealed that the mortality prediction model can be different for males and females with diverse predictors. Patients were classified into four clusters of mortality risk and identified the patients at the highest risk of mortality, which accentuated the most significant predictors correlating with mortality. Conclusion An ML model for predicting mortality among hospitalized COVID-19 patients was developed considering the interactions between factors that may reduce the complexity of clinical decision-making processes. The most predictive factors related to patient mortality were identified by assessing and classifying patients into different groups based on their sex and mortality risk (low-, moderate-, and high-risk groups).
Collapse
Affiliation(s)
| | - Haniyeh Rafiepoor
- Cancer Biology Research Center, Cancer Institute, Tehran University of Medical Sciences, Tehran, Iran
| | - Kazem Zendehdel
- Cancer Biology Research Center, Cancer Institute, Tehran University of Medical Sciences, Tehran, Iran
- Cancer Research Center, Cancer Institute, Tehran University of Medical Sciences, Tehran, Iran
| | - Monireh Sadat Seyyedsalehi
- Cancer Research Center, Cancer Institute, Tehran University of Medical Sciences, Tehran, Iran
- Department of Medical and Surgical Sciences, University of Bologna, Bologna, Italy
| | - Azin Nahvijou
- Cancer Research Center, Cancer Institute, Tehran University of Medical Sciences, Tehran, Iran
| | - Farshad Allameh
- Gastroenterology Ward, Imam Khomeini Hospital Complex (IKHC), Tehran University of Medical Sciences, Tehran, Iran
| | - Saeid Amanpour
- Cancer Biology Research Center, Cancer Institute, Tehran University of Medical Sciences, Tehran, Iran
| |
Collapse
|
212
|
Snell KIE, Levis B, Damen JAA, Dhiman P, Debray TPA, Hooft L, Reitsma JB, Moons KGM, Collins GS, Riley RD. Transparent reporting of multivariable prediction models for individual prognosis or diagnosis: checklist for systematic reviews and meta-analyses (TRIPOD-SRMA). BMJ 2023; 381:e073538. [PMID: 37137496 PMCID: PMC10155050 DOI: 10.1136/bmj-2022-073538] [Citation(s) in RCA: 23] [Impact Index Per Article: 23.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 02/22/2023] [Indexed: 05/05/2023]
Affiliation(s)
- Kym I E Snell
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham B15 2TT, UK
| | - Brooke Levis
- Centre for Prognosis Research, School of Medicine, Keele University, Keele, UK
| | - Johanna A A Damen
- Cochrane Netherlands, Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Paula Dhiman
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
- NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Thomas P A Debray
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Lotty Hooft
- Cochrane Netherlands, Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Johannes B Reitsma
- Cochrane Netherlands, Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Karel G M Moons
- Cochrane Netherlands, Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
- NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Richard D Riley
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham B15 2TT, UK
| |
Collapse
|
213
|
Waksman O, Choi D, Mar P, Chen Q, Cho DJ, Kim H, Smith RL, Goonewardena SN, Rosenson RS. Association of blood viscosity and device-free days among hospitalized patients with COVID-19. J Intensive Care 2023; 11:17. [PMID: 37131249 PMCID: PMC10153022 DOI: 10.1186/s40560-023-00665-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Accepted: 04/17/2023] [Indexed: 05/04/2023] Open
Abstract
BACKGROUND Increased estimated whole blood viscosity (eWBV) predicts higher mortality in patients hospitalized for coronavirus disease 2019 (COVID-19). This study assesses whether eWBV is an early predictor of non-fatal outcomes among patients hospitalized for acute COVID-19 infection. METHODS This retrospective cohort study included 9278 hospitalized COVID-19 patients diagnosed within 48 h of admission between February 27, 2020 to November 20, 2021 within the Mount Sinai Health System in New York City. Patients with missing values for major covariates, discharge information, and those who failed to meet the criteria for the non-Newtonian blood model were excluded. 5621 participants were included in the main analysis. Additional analyses were performed separately for 4352 participants who had measurements of white blood cell count, C-reactive protein and D-dimer. Participants were divided into quartiles based on estimated high-shear blood viscosity (eHSBV) and estimated low-shear blood viscosity (eLSBV). Blood viscosity was calculated using the Walburn-Schneck model. The primary outcome was evaluated as an ordinal scale indicating the number of days free of respiratory organ support through day 21, and those who died in-hospital were assigned a value of -1. Multivariate cumulative logistic regression was conducted to evaluate the association between quartiles of eWBV and events. RESULTS Among 5621 participants, 3459 (61.5%) were male with mean age of 63.2 (SD 17.1) years. The linear modeling yielded an adjusted odds ratio (aOR) of 0.68 (95% CI 0.59-0.79, p value < 0.001) per 1 centipoise increase in eHSBV. CONCLUSIONS Among hospitalized patients with COVID-19, elevated eHSBV and eLSBV at presentation were associated with an increased need for respiratory organ support at 21 days. These findings are highly relevant, as they demonstrate the utility of eWBV in identifying hospitalized patients with acute COVID-19 infection at increased risk for non-fatal outcomes in early stages of the disease.
Collapse
Affiliation(s)
- Ori Waksman
- Metabolism and Lipids Unit, Cardiovascular Institute, Marie-Josee and Henry R Kravis Center for Cardiovascular Health, Mount Sinai Heart, Icahn School of Medicine at Mount Sinai, The Mount Sinai Medical Center, One Gustave L. Levy Place, Box 1030, New York, NY, 10029, USA
| | - Daein Choi
- Metabolism and Lipids Unit, Cardiovascular Institute, Marie-Josee and Henry R Kravis Center for Cardiovascular Health, Mount Sinai Heart, Icahn School of Medicine at Mount Sinai, The Mount Sinai Medical Center, One Gustave L. Levy Place, Box 1030, New York, NY, 10029, USA
- Department of Medicine, Mount Sinai Beth Israel, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Phyu Mar
- Metabolism and Lipids Unit, Cardiovascular Institute, Marie-Josee and Henry R Kravis Center for Cardiovascular Health, Mount Sinai Heart, Icahn School of Medicine at Mount Sinai, The Mount Sinai Medical Center, One Gustave L. Levy Place, Box 1030, New York, NY, 10029, USA
| | - Qinzhong Chen
- Metabolism and Lipids Unit, Cardiovascular Institute, Marie-Josee and Henry R Kravis Center for Cardiovascular Health, Mount Sinai Heart, Icahn School of Medicine at Mount Sinai, The Mount Sinai Medical Center, One Gustave L. Levy Place, Box 1030, New York, NY, 10029, USA
| | | | | | | | - Sascha N Goonewardena
- Division of Cardiovascular Medicine, Department of Internal Medicine, University of Michigan, Ann Arbor, MI, USA
| | - Robert S Rosenson
- Metabolism and Lipids Unit, Cardiovascular Institute, Marie-Josee and Henry R Kravis Center for Cardiovascular Health, Mount Sinai Heart, Icahn School of Medicine at Mount Sinai, The Mount Sinai Medical Center, One Gustave L. Levy Place, Box 1030, New York, NY, 10029, USA.
| |
Collapse
|
214
|
Dabbagh R, Jamal A, Bhuiyan Masud JH, Titi MA, Amer YS, Khayat A, Alhazmi TS, Hneiny L, Baothman FA, Alkubeyyer M, Khan SA, Temsah MH. Harnessing Machine Learning in Early COVID-19 Detection and Prognosis: A Comprehensive Systematic Review. Cureus 2023; 15:e38373. [PMID: 37265897 PMCID: PMC10230599 DOI: 10.7759/cureus.38373] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/30/2023] [Indexed: 06/03/2023] Open
Abstract
During the early phase of the COVID-19 pandemic, reverse transcriptase-polymerase chain reaction (RT-PCR) testing faced limitations, prompting the exploration of machine learning (ML) alternatives for diagnosis and prognosis. Providing a comprehensive appraisal of such decision support systems and their use in COVID-19 management can aid the medical community in making informed decisions during the risk assessment of their patients, especially in low-resource settings. Therefore, the objective of this study was to systematically review the studies that predicted the diagnosis of COVID-19 or the severity of the disease using ML. Following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA), we conducted a literature search of MEDLINE (OVID), Scopus, EMBASE, and IEEE Xplore from January 1 to June 31, 2020. The outcomes were COVID-19 diagnosis or prognostic measures such as death, need for mechanical ventilation, admission, and acute respiratory distress syndrome. We included peer-reviewed observational studies, clinical trials, research letters, case series, and reports. We extracted data about the study's country, setting, sample size, data source, dataset, diagnostic or prognostic outcomes, prediction measures, type of ML model, and measures of diagnostic accuracy. Bias was assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). This study was registered in the International Prospective Register of Systematic Reviews (PROSPERO), with the number CRD42020197109. The final records included for data extraction were 66. Forty-three (64%) studies used secondary data. The majority of studies were from Chinese authors (30%). Most of the literature (79%) relied on chest imaging for prediction, while the remainder used various laboratory indicators, including hematological, biochemical, and immunological markers. Thirteen studies explored predicting COVID-19 severity, while the rest predicted diagnosis. Seventy percent of the articles used deep learning models, while 30% used traditional ML algorithms. Most studies reported high sensitivity, specificity, and accuracy for the ML models (exceeding 90%). The overall concern about the risk of bias was "unclear" in 56% of the studies. This was mainly due to concerns about selection bias. ML may help identify COVID-19 patients in the early phase of the pandemic, particularly in the context of chest imaging. Although these studies reflect that these ML models exhibit high accuracy, the novelty of these models and the biases in dataset selection make using them as a replacement for the clinicians' cognitive decision-making questionable. Continued research is needed to enhance the robustness and reliability of ML systems in COVID-19 diagnosis and prognosis.
Collapse
Affiliation(s)
- Rufaidah Dabbagh
- Family & Community Medicine Department, College of Medicine, King Saud University, Riyadh, SAU
| | - Amr Jamal
- Family & Community Medicine Department, College of Medicine, King Saud University, Riyadh, SAU
- Research Chair for Evidence-Based Health Care and Knowledge Translation, Family and Community Medicine Department, College of Medicine, King Saud University, Riyadh, SAU
| | | | - Maher A Titi
- Quality Management Department, King Saud University Medical City, Riyadh, SAU
- Research Chair for Evidence-Based Health Care and Knowledge Translation, Family and Community Medicine Department, College of Medicine, King Saud University, Riyadh, SAU
| | - Yasser S Amer
- Pediatrics, Quality Management Department, King Saud University Medical City, Riyadh, SAU
- Research Chair for Evidence-Based Health Care and Knowledge Translation, Family and Community Medicine Department, College of Medicine, King Saud University, Riyadh, SAU
| | - Afnan Khayat
- Health Information Management Department, Prince Sultan Military College of Health Sciences, Al Dhahran, SAU
| | - Taha S Alhazmi
- Family & Community Medicine Department, College of Medicine, King Saud University, Riyadh, SAU
| | - Layal Hneiny
- Medicine, Wegner Health Sciences Library, University of South Dakota, Vermillion, USA
| | - Fatmah A Baothman
- Department of Information Systems, King Abdulaziz University, Jeddah, SAU
| | | | - Samina A Khan
- School of Computer Sciences, Universiti Sains Malaysia, Penang, MYS
| | - Mohamad-Hani Temsah
- Pediatric Intensive Care Unit, Department of Pediatrics, King Saud University, Riyadh, SAU
| |
Collapse
|
215
|
Dhiman P, Ma J, Andaur Navarro CL, Speich B, Bullock G, Damen JAA, Hooft L, Kirtley S, Riley RD, Van Calster B, Moons KGM, Collins GS. Overinterpretation of findings in machine learning prediction model studies in oncology: a systematic review. J Clin Epidemiol 2023; 157:120-133. [PMID: 36935090 DOI: 10.1016/j.jclinepi.2023.03.012] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2022] [Revised: 03/03/2023] [Accepted: 03/14/2023] [Indexed: 03/19/2023]
Abstract
OBJECTIVES In biomedical research, spin is the overinterpretation of findings, and it is a growing concern. To date, the presence of spin has not been evaluated in prognostic model research in oncology, including studies developing and validating models for individualized risk prediction. STUDY DESIGN AND SETTING We conducted a systematic review, searching MEDLINE and EMBASE for oncology-related studies that developed and validated a prognostic model using machine learning published between 1st January, 2019, and 5th September, 2019. We used existing spin frameworks and described areas of highly suggestive spin practices. RESULTS We included 62 publications (including 152 developed models; 37 validated models). Reporting was inconsistent between methods and the results in 27% of studies due to additional analysis and selective reporting. Thirty-two studies (out of 36 applicable studies) reported comparisons between developed models in their discussion and predominantly used discrimination measures to support their claims (78%). Thirty-five studies (56%) used an overly strong or leading word in their title, abstract, results, discussion, or conclusion. CONCLUSION The potential for spin needs to be considered when reading, interpreting, and using studies that developed and validated prognostic models in oncology. Researchers should carefully report their prognostic model research using words that reflect their actual results and strength of evidence.
Collapse
Affiliation(s)
- Paula Dhiman
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK; NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK.
| | - Jie Ma
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Constanza L Andaur Navarro
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Benjamin Speich
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK; Meta-Research Centre, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland
| | - Garrett Bullock
- Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Johanna A A Damen
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Lotty Hooft
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Shona Kirtley
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Richard D Riley
- Centre for Prognosis Research, School of Medicine, Keele University, Staffordshire, UK, ST5 5BG
| | - Ben Van Calster
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium; Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands; EPI-centre, KU Leuven, Leuven, Belgium
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK; NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| |
Collapse
|
216
|
Wilmes N, Hendriks CWE, Viets CTA, Cornelissen SJWM, van Mook WNKA, Cox-Brinkman J, Celi LA, Martinez-Martin N, Gichoya JW, Watkins C, Bakhshi-Raiez F, Wynants L, van der Horst ICC, van Bussel BCT. Structural under-reporting of informed consent, data handling and sharing, ethical approval, and application of Open Science principles as proxies for study quality conduct in COVID-19 research: a systematic scoping review. BMJ Glob Health 2023; 8:e012007. [PMID: 37257937 PMCID: PMC10254958 DOI: 10.1136/bmjgh-2023-012007] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2023] [Accepted: 04/13/2023] [Indexed: 06/02/2023] Open
Abstract
BACKGROUND The COVID-19 pandemic required science to provide answers rapidly to combat the outbreak. Hence, the reproducibility and quality of conducting research may have been threatened, particularly regarding privacy and data protection, in varying ways around the globe. The objective was to investigate aspects of reporting informed consent and data handling as proxies for study quality conduct. METHODS A systematic scoping review was performed by searching PubMed and Embase. The search was performed on November 8th, 2020. Studies with hospitalised patients diagnosed with COVID-19 over 18 years old were eligible for inclusion. With a focus on informed consent, data were extracted on the study design, prestudy protocol registration, ethical approval, data anonymisation, data sharing and data transfer as proxies for study quality. For reasons of comparison, data regarding country income level, study location and journal impact factor were also collected. RESULTS 972 studies were included. 21.3% of studies reported informed consent, 42.6% reported waivers of consent, 31.4% did not report consent information and 4.7% mentioned other types of consent. Informed consent reporting was highest in clinical trials (94.6%) and lowest in retrospective cohort studies (15.0%). The reporting of consent versus no consent did not differ significantly by journal impact factor (p=0.159). 16.8% of studies reported a prestudy protocol registration or design. Ethical approval was described in 90.9% of studies. Information on anonymisation was provided in 17.0% of studies. In 257 multicentre studies, 1.2% reported on data sharing agreements, and none reported on Findable, Accessible, Interoperable and Reusable data principles. 1.2% reported on open data. Consent was most often reported in the Middle East (42.4%) and least often in North America (4.7%). Only one report originated from a low-income country. DISCUSSION Informed consent and aspects of data handling and sharing were under-reported in publications concerning COVID-19 and differed between countries, which strains study quality conduct when in dire need of answers.
Collapse
Affiliation(s)
- Nick Wilmes
- Deparment of Intensive Care Medicine, Maastricht University Medical Centre, Maastricht, The Netherlands
- Cardiovascular research institute Maastricht (CARIM), Maastricht University, Maastricht, The Netherlands
| | - Charlotte W E Hendriks
- Deparment of Intensive Care Medicine, Maastricht University Medical Centre, Maastricht, The Netherlands
| | - Caspar T A Viets
- Deparment of Intensive Care Medicine, Maastricht University Medical Centre, Maastricht, The Netherlands
| | - Simon J W M Cornelissen
- Deparment of Intensive Care Medicine, Maastricht University Medical Centre, Maastricht, The Netherlands
| | - Walther N K A van Mook
- Deparment of Intensive Care Medicine, Maastricht University Medical Centre, Maastricht, The Netherlands
- Academy for Postgraduate Medical Training, Maastricht University Medical Center, Maastricht, The Netherlands
- School of Health Professions Education, Maastricht University, Maastricht, The Netherlands
| | - Josanne Cox-Brinkman
- Department of Health Law, Maastricht University Medical Centre, Maastricht, The Netherlands
| | - Leo A Celi
- Division of Pulmonary, Critical Care and Sleep Medicine, Beth Israel Deaconess Medical Centre, Boston, Massachusetts, USA
- Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA
- Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, USA
| | - Nicole Martinez-Martin
- Department of Pediatrics, Stanford Center for Biomedical Ethics, Stanford University, San Jose, California, USA
| | - Judy W Gichoya
- Department of Radiology and Imaging Sciences, Emory University School of Medicine, Emory University, Atlanta, Florida, USA
| | - Craig Watkins
- School of Journalism and Media, University of Texas, Austin, Texas, USA
| | - Ferishta Bakhshi-Raiez
- Department of Medical Informatics, Amsterdam University Medical Centre, Amsterdam and Amsterdam Public Health, Quality of care The Netherlands, Amsterdam, The Netherlands
| | - Laure Wynants
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium
- Care and Public Health Research Institute, Maastricht University, Maastricht, The Netherlands
| | - Iwan C C van der Horst
- Deparment of Intensive Care Medicine, Maastricht University Medical Centre, Maastricht, The Netherlands
- Cardiovascular research institute Maastricht (CARIM), Maastricht University, Maastricht, The Netherlands
| | - Bas C T van Bussel
- Deparment of Intensive Care Medicine, Maastricht University Medical Centre, Maastricht, The Netherlands
- Cardiovascular research institute Maastricht (CARIM), Maastricht University, Maastricht, The Netherlands
- Care and Public Health Research Institute, Maastricht University, Maastricht, The Netherlands
| |
Collapse
|
217
|
Truchot A, Raynaud M, Kamar N, Naesens M, Legendre C, Delahousse M, Thaunat O, Buchler M, Crespo M, Linhares K, Orandi BJ, Akalin E, Pujol GS, Silva HT, Gupta G, Segev DL, Jouven X, Bentall AJ, Stegall MD, Lefaucheur C, Aubert O, Loupy A. Machine learning does not outperform traditional statistical modelling for kidney allograft failure prediction. Kidney Int 2023; 103:936-948. [PMID: 36572246 DOI: 10.1016/j.kint.2022.12.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Revised: 11/04/2022] [Accepted: 12/15/2022] [Indexed: 12/24/2022]
Abstract
Machine learning (ML) models have recently shown potential for predicting kidney allograft outcomes. However, their ability to outperform traditional approaches remains poorly investigated. Therefore, using large cohorts of kidney transplant recipients from 14 centers worldwide, we developed ML-based prediction models for kidney allograft survival and compared their prediction performances to those achieved by a validated Cox-Based Prognostication System (CBPS). In a French derivation cohort of 4000 patients, candidate determinants of allograft failure including donor, recipient and transplant-related parameters were used as predictors to develop tree-based models (RSF, RSF-ERT, CIF), Support Vector Machine models (LK-SVM, AK-SVM) and a gradient boosting model (XGBoost). Models were externally validated with cohorts of 2214 patients from Europe, 1537 from North America, and 671 from South America. Among these 8422 kidney transplant recipients, 1081 (12.84%) lost their grafts after a median post-transplant follow-up time of 6.25 years (Inter Quartile Range 4.33-8.73). At seven years post-risk evaluation, the ML models achieved a C-index of 0.788 (95% bootstrap percentile confidence interval 0.736-0.833), 0.779 (0.724-0.825), 0.786 (0.735-0.832), 0.527 (0.456-0.602), 0.704 (0.648-0.759) and 0.767 (0.711-0.815) for RSF, RSF-ERT, CIF, LK-SVM, AK-SVM and XGBoost respectively, compared with 0.808 (0.792-0.829) for the CBPS. In validation cohorts, ML models' discrimination performances were in a similar range of those of the CBPS. Calibrations of the ML models were similar or less accurate than those of the CBPS. Thus, when using a transparent methodological pipeline in validated international cohorts, ML models, despite overall good performances, do not outperform a traditional CBPS in predicting kidney allograft failure. Hence, our current study supports the continued use of traditional statistical approaches for kidney graft prognostication.
Collapse
Affiliation(s)
- Agathe Truchot
- Université de Paris, INSERM, PARCC, Paris Translational Research Centre for Organ Transplantation, Paris, France
| | - Marc Raynaud
- Université de Paris, INSERM, PARCC, Paris Translational Research Centre for Organ Transplantation, Paris, France
| | - Nassim Kamar
- Université Paul Sabatier, INSERM, Department of Nephrology and Organ Transplantation, CHU Rangueil and Purpan, Toulouse, France
| | - Maarten Naesens
- Department of Microbiology, Immunology and Transplantation, KU Leuven, Leuven, Belgium
| | - Christophe Legendre
- Université de Paris, INSERM, PARCC, Paris Translational Research Centre for Organ Transplantation, Paris, France; Kidney Transplant Department, Necker Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
| | - Michel Delahousse
- Department of Transplantation, Nephrology and Clinical Immunology, Foch Hospital, Suresnes, France
| | - Olivier Thaunat
- Department of Transplantation, Nephrology and Clinical Immunology, Hospices Civils de Lyon, Lyon, France
| | - Matthias Buchler
- Nephrology and Immunology Department, Bretonneau Hospital, Tours, France
| | - Marta Crespo
- Department of Nephrology, Hospital del Mar Barcelona, Barcelona, Spain
| | - Kamilla Linhares
- Hospital do Rim, Escola Paulista de Medicina, Universidade Federal de São Paulo, São Paulo, Brazil
| | - Babak J Orandi
- University of Alabama at Birmingham Heersink School of Medicine, Birmingham, Alabama, USA
| | - Enver Akalin
- Renal Division, Montefiore Medical Centre, Kidney Transplantation Program, Albert Einstein College of Medicine, New York, New York, USA
| | - Gervacio Soler Pujol
- Unidad de Trasplante Renopancreas, Centro de Educacion Medica e Investigaciones Clinicas Buenos Aires, Buenos Aires, Argentina
| | - Helio Tedesco Silva
- Hospital do Rim, Escola Paulista de Medicina, Universidade Federal de São Paulo, São Paulo, Brazil
| | - Gaurav Gupta
- Division of Nephrology, Department of Internal Medicine, Virginia Commonwealth University School of Medicine, Richmond, Virginia, USA
| | - Dorry L Segev
- Department of Surgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
| | - Xavier Jouven
- Université de Paris, INSERM, PARCC, Paris Translational Research Centre for Organ Transplantation, Paris, France; Cardiology Department, European Georges Pompidou Hospital, Paris, France
| | - Andrew J Bentall
- William J von Liebig Centre for Transplantation and Clinical Regeneration, Mayo Clinic, Rochester, Minnesota, USA
| | - Mark D Stegall
- William J von Liebig Centre for Transplantation and Clinical Regeneration, Mayo Clinic, Rochester, Minnesota, USA
| | - Carmen Lefaucheur
- Université de Paris, INSERM, PARCC, Paris Translational Research Centre for Organ Transplantation, Paris, France; Kidney Transplant Department, Saint-Louis Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
| | - Olivier Aubert
- Université de Paris, INSERM, PARCC, Paris Translational Research Centre for Organ Transplantation, Paris, France; Kidney Transplant Department, Necker Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
| | - Alexandre Loupy
- Université de Paris, INSERM, PARCC, Paris Translational Research Centre for Organ Transplantation, Paris, France; Kidney Transplant Department, Necker Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France.
| |
Collapse
|
218
|
Choi Y, Yu W, Nagarajan MB, Teng P, Goldin JG, Raman SS, Enzmann DR, Kim GHJ, Brown MS. Translating AI to Clinical Practice: Overcoming Data Shift with Explainability. Radiographics 2023; 43:e220105. [PMID: 37104124 PMCID: PMC10190133 DOI: 10.1148/rg.220105] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2022] [Revised: 09/09/2022] [Accepted: 09/23/2022] [Indexed: 04/28/2023]
Abstract
To translate artificial intelligence (AI) algorithms into clinical practice requires generalizability of models to real-world data. One of the main obstacles to generalizability is data shift, a data distribution mismatch between model training and real environments. Explainable AI techniques offer tools to detect and mitigate the data shift problem and develop reliable AI for clinical practice. Most medical AI is trained with datasets gathered from limited environments, such as restricted disease populations and center-dependent acquisition conditions. The data shift that commonly exists in the limited training set often causes a significant performance decrease in the deployment environment. To develop a medical application, it is important to detect potential data shift and its impact on clinical translation. During AI training stages, from premodel analysis to in-model and post hoc explanations, explainability can play a key role in detecting model susceptibility to data shift, which is otherwise hidden because the test data have the same biased distribution as the training data. Performance-based model assessments cannot effectively distinguish the model overfitting to training data bias without enriched test sets from external environments. In the absence of such external data, explainability techniques can aid in translating AI to clinical practice as a tool to detect and mitigate potential failures due to data shift. ©RSNA, 2023 Quiz questions for this article are available in the supplemental material.
Collapse
Affiliation(s)
- Youngwon Choi
- From the Center for Computer Vision and Imaging Biomarkers, 924
Westwood Blvd, Los Angeles, CA 90024 (Y.C., W.Y., M.B.N., P.T., J.G.G.,
G.H.J.K., M.S.B.); and Department of Radiology, University of
California–Los Angeles, Los Angeles, Calif (Y.C., W.Y., M.B.N., P.T.,
J.G.G., S.S.R., D.R.E., G.H.J.K., M.S.B.)
| | - Wenxi Yu
- From the Center for Computer Vision and Imaging Biomarkers, 924
Westwood Blvd, Los Angeles, CA 90024 (Y.C., W.Y., M.B.N., P.T., J.G.G.,
G.H.J.K., M.S.B.); and Department of Radiology, University of
California–Los Angeles, Los Angeles, Calif (Y.C., W.Y., M.B.N., P.T.,
J.G.G., S.S.R., D.R.E., G.H.J.K., M.S.B.)
| | - Mahesh B. Nagarajan
- From the Center for Computer Vision and Imaging Biomarkers, 924
Westwood Blvd, Los Angeles, CA 90024 (Y.C., W.Y., M.B.N., P.T., J.G.G.,
G.H.J.K., M.S.B.); and Department of Radiology, University of
California–Los Angeles, Los Angeles, Calif (Y.C., W.Y., M.B.N., P.T.,
J.G.G., S.S.R., D.R.E., G.H.J.K., M.S.B.)
| | - Pangyu Teng
- From the Center for Computer Vision and Imaging Biomarkers, 924
Westwood Blvd, Los Angeles, CA 90024 (Y.C., W.Y., M.B.N., P.T., J.G.G.,
G.H.J.K., M.S.B.); and Department of Radiology, University of
California–Los Angeles, Los Angeles, Calif (Y.C., W.Y., M.B.N., P.T.,
J.G.G., S.S.R., D.R.E., G.H.J.K., M.S.B.)
| | - Jonathan G. Goldin
- From the Center for Computer Vision and Imaging Biomarkers, 924
Westwood Blvd, Los Angeles, CA 90024 (Y.C., W.Y., M.B.N., P.T., J.G.G.,
G.H.J.K., M.S.B.); and Department of Radiology, University of
California–Los Angeles, Los Angeles, Calif (Y.C., W.Y., M.B.N., P.T.,
J.G.G., S.S.R., D.R.E., G.H.J.K., M.S.B.)
| | - Steven S. Raman
- From the Center for Computer Vision and Imaging Biomarkers, 924
Westwood Blvd, Los Angeles, CA 90024 (Y.C., W.Y., M.B.N., P.T., J.G.G.,
G.H.J.K., M.S.B.); and Department of Radiology, University of
California–Los Angeles, Los Angeles, Calif (Y.C., W.Y., M.B.N., P.T.,
J.G.G., S.S.R., D.R.E., G.H.J.K., M.S.B.)
| | - Dieter R. Enzmann
- From the Center for Computer Vision and Imaging Biomarkers, 924
Westwood Blvd, Los Angeles, CA 90024 (Y.C., W.Y., M.B.N., P.T., J.G.G.,
G.H.J.K., M.S.B.); and Department of Radiology, University of
California–Los Angeles, Los Angeles, Calif (Y.C., W.Y., M.B.N., P.T.,
J.G.G., S.S.R., D.R.E., G.H.J.K., M.S.B.)
| | - Grace Hyun J. Kim
- From the Center for Computer Vision and Imaging Biomarkers, 924
Westwood Blvd, Los Angeles, CA 90024 (Y.C., W.Y., M.B.N., P.T., J.G.G.,
G.H.J.K., M.S.B.); and Department of Radiology, University of
California–Los Angeles, Los Angeles, Calif (Y.C., W.Y., M.B.N., P.T.,
J.G.G., S.S.R., D.R.E., G.H.J.K., M.S.B.)
| | - Matthew S. Brown
- From the Center for Computer Vision and Imaging Biomarkers, 924
Westwood Blvd, Los Angeles, CA 90024 (Y.C., W.Y., M.B.N., P.T., J.G.G.,
G.H.J.K., M.S.B.); and Department of Radiology, University of
California–Los Angeles, Los Angeles, Calif (Y.C., W.Y., M.B.N., P.T.,
J.G.G., S.S.R., D.R.E., G.H.J.K., M.S.B.)
| |
Collapse
|
219
|
Richter T, Tesch F, Schmitt J, Koschel D, Kolditz M. Validation of the qSOFA and CRB-65 in SARS-CoV-2-infected community-acquired pneumonia. ERJ Open Res 2023; 9:00168-2023. [PMID: 37337510 PMCID: PMC10105511 DOI: 10.1183/23120541.00168-2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Accepted: 04/05/2023] [Indexed: 06/21/2023] Open
Abstract
Rationale Prognostic accuracy of the quick sequential organ failure assessment (qSOFA) and CRB-65 (confusion, respiratory rate, blood pressure and age (≥65 years)) risk scores have not been widely evaluated in patients with SARS-CoV-2-positive compared to SARS-CoV-2-negative community-acquired pneumonia (CAP). The aim of the present study was to validate the qSOFA(-65) and CRB-65 scores in a large cohort of SARS-CoV-2-positive and SARS-CoV-2-negative CAP patients. Methods We included all cases with CAP hospitalised in 2020 from the German nationwide mandatory quality assurance programme and compared cases with SARS-CoV-2 infection to cases without. We excluded cases with unclear SARS-CoV-2 infection state, transferred to another hospital or on mechanical ventilation during admission. Predefined outcomes were hospital mortality and need for mechanical ventilation. Results Among 68 594 SARS-CoV-2-positive patients, hospital mortality (22.7%) and mechanical ventilation (14.9%) were significantly higher when compared to 167 880 SARS-CoV-2-negative patients (15.7% and 9.2%, respectively). All CRB-65 and qSOFA criteria were associated with both outcomes, and age dominated mortality prediction in SARS-CoV-2 (risk ratio >9). Scores including the age criterion had higher area under the curve (AUCs) for mortality in SARS-CoV-2-positive patients (e.g. CRB-65 AUC 0.76) compared to SARS-CoV-2 negative patients (AUC 0.68), and negative predictive value was highest for qSOFA-65=0 (98.2%). Sensitivity for mechanical ventilation prediction was poor with all scores (AUCs 0.59-0.62), and negative predictive values were insufficient (qSOFA-65=0 missed 1490 out of 10 198 patients (∼15%) with mechanical ventilation). Results were similar when excluding frail and palliative patients. Conclusions Hospital mortality and mechanical ventilation rates were higher in SARS-CoV-2-positive than SARS-CoV-2-negative CAP. For SARS-CoV-2-positive CAP, the CRB-65 and qSOFA-65 scores showed adequate prediction of mortality but not of mechanical ventilation.
Collapse
Affiliation(s)
- Tina Richter
- Division of Pulmonology, Medical Department I, University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| | - Falko Tesch
- Dresden University Centre for Evidence-Based Healthcare, University Hospital and Faculty of Medicine Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| | - Jochen Schmitt
- Dresden University Centre for Evidence-Based Healthcare, University Hospital and Faculty of Medicine Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| | - Dirk Koschel
- Division of Pulmonology, Medical Department I, University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| | - Martin Kolditz
- Division of Pulmonology, Medical Department I, University Hospital Carl Gustav Carus, Technische Universität Dresden, Dresden, Germany
| |
Collapse
|
220
|
Siavoshi F, Safavi-Naini SAA, Shirzadeh Barough S, Azizmohammad Looha M, Hatamabadi H, Ommi D, Jalili Khoshnoud R, Fatemi A, Pourhoseingholi MA. On-admission and dynamic trend of laboratory profiles as prognostic biomarkers in COVID-19 inpatients. Sci Rep 2023; 13:6993. [PMID: 37117397 PMCID: PMC10144885 DOI: 10.1038/s41598-023-34166-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Accepted: 04/25/2023] [Indexed: 04/30/2023] Open
Abstract
This large-scale study aimed to investigate the trend of laboratory tests of patients with COVID-19. Hospitalized confirmed and probable COVID-19 patients in three general hospitals were examined from March 20, 2020, to June 18, 2021. The confirmed and probable COVID-19 patients with known outcomes and valid laboratory results were included. The least absolute shrinkage and selection operator (LASSO) and Cox regression were used to select admittance prognostic features. Parallel Pairwise Comparison of mortality versus survival was used to examine the trend of markers. In the final cohort, 11,944 patients were enrolled, with an in-hospital mortality rate of 21.8%, mean age of 59.4 ± 18.0, and a male-to-female ratio of 1.3. Abnormal admittance level of white blood cells, neutrophils, lymphocytes, mean cellular volume, urea, creatinine, bilirubin, creatine kinase-myoglobin binding, lactate dehydrogenase (LDH), Troponin, c-reactive protein (CRP), potassium, and creatinine phosphokinase reduced the survival of COVID-19 inpatients. Moreover, the trend analysis showed lymphocytes, platelet, urea, CRP, alanine transaminase (ALT), and LDH have a dissimilar trend in non-survivors compared to survived patients. This study proposed a novel approach to find serial laboratory markers. Serial examination of platelet count, creatinine, CRP, LDH, and ALT can guide healthcare professionals in finding patients at risk of deterioration.
Collapse
Affiliation(s)
- Fatemeh Siavoshi
- Basic and Molecular Epidemiology of Gastrointestinal Disorders Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Seyed Amir Ahmad Safavi-Naini
- Basic and Molecular Epidemiology of Gastrointestinal Disorders Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Siavash Shirzadeh Barough
- Basic and Molecular Epidemiology of Gastrointestinal Disorders Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Mehdi Azizmohammad Looha
- Basic and Molecular Epidemiology of Gastrointestinal Disorders Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Hamidreza Hatamabadi
- Safety Promotion and Injury Prevention Research Center, Imam Hossein Hospital, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Davood Ommi
- Department of Anesthesiology, School of Medicine, Shohada-e-Tajrish Hospital, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Reza Jalili Khoshnoud
- Department of Neurosurgery, School of Medicine, Shohada-e-Tajrish Hospital, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Alireza Fatemi
- Men's Health and Reproductive Health Research Center, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
| | - Mohamad Amin Pourhoseingholi
- Basic and Molecular Epidemiology of Gastrointestinal Disorders Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
| |
Collapse
|
221
|
Li QY, An ZY, Pan ZH, Wang ZZ, Wang YR, Zhang XG, Shen N. Severe/critical COVID-19 early warning system based on machine learning algorithms using novel imaging scores. World J Clin Cases 2023; 11:2716-2728. [PMID: 37214568 PMCID: PMC10198108 DOI: 10.12998/wjcc.v11.i12.2716] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/05/2022] [Revised: 02/12/2023] [Accepted: 03/17/2023] [Indexed: 04/25/2023] Open
Abstract
BACKGROUND Early identification of severe/critical coronavirus disease 2019 (COVID-19) is crucial for timely treatment and intervention. Chest computed tomography (CT) score has been shown to be a significant factor in the diagnosis and treatment of pneumonia, however, there is currently a lack of effective early warning systems for severe/critical COVID-19 based on dynamic CT evolution.
AIM To develop a severe/critical COVID-19 prediction model using a combination of imaging scores, clinical features, and biomarker levels.
METHODS This study used an improved scoring system to extract and describe the chest CT characteristics of COVID-19 patients. The study also took into consideration the general clinical indicators such as dyspnea, oxygen saturation, alternative lengthening of telomeres (ALT), and androgen suppression treatment (AST), which are commonly associated with severe/critical COVID-19 cases. The study employed lasso regression to evaluate and rank the significance of different disease characteristics.
RESULTS The results showed that blood oxygen saturation, ALT, IL-6/IL-10, combined score, ground glass opacity score, age, crazy paving mode score, qsofa, AST, and overall lung involvement score were key factors in predicting severe/critical COVID-19 cases. The study established a COVID-19 severe/critical early warning system using various machine learning algorithms, including XGBClassifier, Logistic Regression, MLPClassifier, RandomForestClassifier, and AdaBoost Classifier. The study concluded that the prediction model based on the improved CT score and machine learning algorithms is a feasible method for early detection of severe/critical COVID-19 evolution.
CONCLUSION The findings of this study suggest that a prediction model based on improved CT scores and machine learning algorithms is effective in detecting the early warning signals of severe/critical COVID-19.
Collapse
Affiliation(s)
- Qiu-Yu Li
- Department of Respiratory and Critical Care Medicine, Peking University Third Hospital, Beijing 100191, China
| | - Zhuo-Yu An
- Department of Education, Peking University People’s Hospital, Beijing 100044, China
| | - Zi-Han Pan
- Department of Respiratory and Critical Care Medicine, Peking University Third Hospital, Beijing 100191, China
| | - Zi-Zhen Wang
- Department of Education, China-Japan Friendship Hospital, Beijing 100029, China
| | - Yi-Ren Wang
- Department of Education, Peking University People’s Hospital, Beijing 100044, China
| | - Xi-Gong Zhang
- Department of Education, Beijing Jishuitan Hospital, Beijing 100096, China
| | - Ning Shen
- Department of Respiratory and Critical Care Medicine, Peking University Third Hospital, Beijing 100191, China
| |
Collapse
|
222
|
Yasrebi-de Kom IAR, Dongelmans DA, de Keizer NF, Jager KJ, Schut MC, Abu-Hanna A, Klopotowska JE. Electronic health record-based prediction models for in-hospital adverse drug event diagnosis or prognosis: a systematic review. J Am Med Inform Assoc 2023; 30:978-988. [PMID: 36805926 PMCID: PMC10114128 DOI: 10.1093/jamia/ocad014] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Revised: 01/13/2023] [Accepted: 02/01/2023] [Indexed: 02/22/2023] Open
Abstract
OBJECTIVE We conducted a systematic review to characterize and critically appraise developed prediction models based on structured electronic health record (EHR) data for adverse drug event (ADE) diagnosis and prognosis in adult hospitalized patients. MATERIALS AND METHODS We searched the Embase and Medline databases (from January 1, 1999, to July 4, 2022) for articles utilizing structured EHR data to develop ADE prediction models for adult inpatients. For our systematic evidence synthesis and critical appraisal, we applied the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS). RESULTS Twenty-five articles were included. Studies often did not report crucial information such as patient characteristics or the method for handling missing data. In addition, studies frequently applied inappropriate methods, such as univariable screening for predictor selection. Furthermore, the majority of the studies utilized ADE labels that only described an adverse symptom while not assessing causality or utilizing a causal model. None of the models were externally validated. CONCLUSIONS Several challenges should be addressed before the models can be widely implemented, including the adherence to reporting standards and the adoption of best practice methods for model development and validation. In addition, we propose a reorientation of the ADE prediction modeling domain to include causality as a fundamental challenge that needs to be addressed in future studies, either through acquiring ADE labels via formal causality assessments or the usage of adverse event labels in combination with causal prediction modeling.
Collapse
Affiliation(s)
- Izak A R Yasrebi-de Kom
- Amsterdam UMC location University of Amsterdam, Department of Medical Informatics, Amsterdam, The Netherlands
- Amsterdam Public Health, Amsterdam, The Netherlands
| | - Dave A Dongelmans
- Amsterdam Public Health, Amsterdam, The Netherlands
- Amsterdam UMC location University of Amsterdam, Department of Intensive Care Medicine, Amsterdam, The Netherlands
| | - Nicolette F de Keizer
- Amsterdam UMC location University of Amsterdam, Department of Medical Informatics, Amsterdam, The Netherlands
- Amsterdam Public Health, Amsterdam, The Netherlands
| | - Kitty J Jager
- Amsterdam UMC location University of Amsterdam, Department of Medical Informatics, Amsterdam, The Netherlands
- Amsterdam Public Health, Amsterdam, The Netherlands
- Amsterdam Cardiovascular Sciences, Pulmonary Hypertension & Thrombosis, Amsterdam, The Netherlands
| | - Martijn C Schut
- Amsterdam UMC location University of Amsterdam, Department of Medical Informatics, Amsterdam, The Netherlands
- Amsterdam Public Health, Amsterdam, The Netherlands
- Amsterdam UMC location Vrije Universiteit Amsterdam, Department of Clinical Chemistry, Amsterdam, The Netherlands
| | - Ameen Abu-Hanna
- Amsterdam UMC location University of Amsterdam, Department of Medical Informatics, Amsterdam, The Netherlands
- Amsterdam Public Health, Amsterdam, The Netherlands
| | - Joanna E Klopotowska
- Amsterdam UMC location University of Amsterdam, Department of Medical Informatics, Amsterdam, The Netherlands
- Amsterdam Public Health, Amsterdam, The Netherlands
| |
Collapse
|
223
|
Ansell L, Dalla Valle L. A new data integration framework for Covid-19 social media information. Sci Rep 2023; 13:6170. [PMID: 37061597 PMCID: PMC10105535 DOI: 10.1038/s41598-023-33141-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2022] [Accepted: 04/07/2023] [Indexed: 04/17/2023] Open
Abstract
The Covid-19 pandemic presents a serious threat to people's health, resulting in over 250 million confirmed cases and over 5 million deaths globally. To reduce the burden on national health care systems and to mitigate the effects of the outbreak, accurate modelling and forecasting methods for short- and long-term health demand are needed to inform government interventions aiming at curbing the pandemic. Current research on Covid-19 is typically based on a single source of information, specifically on structured historical pandemic data. Other studies are exclusively focused on unstructured online retrieved insights, such as data available from social media. However, the combined use of structured and unstructured information is still uncharted. This paper aims at filling this gap, by leveraging historical and social media information with a novel data integration methodology. The proposed approach is based on vine copulas, which allow us to exploit the dependencies between different sources of information. We apply the methodology to combine structured datasets retrieved from official sources and a big unstructured dataset of information collected from social media. The results show that the combined use of official and online generated information contributes to yield a more accurate assessment of the evolution of the Covid-19 pandemic, compared to the sole use of official data.
Collapse
Affiliation(s)
- Lauren Ansell
- School of Engineering, Computing and Mathematics, University of Plymouth, Plymouth, PL48AA, UK
| | - Luciana Dalla Valle
- School of Engineering, Computing and Mathematics, University of Plymouth, Plymouth, PL48AA, UK.
| |
Collapse
|
224
|
Rao R, Musante CJ, Allen R. A quantitative systems pharmacology model of the pathophysiology and treatment of COVID-19 predicts optimal timing of pharmacological interventions. NPJ Syst Biol Appl 2023; 9:13. [PMID: 37059734 PMCID: PMC10102696 DOI: 10.1038/s41540-023-00269-6] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2022] [Accepted: 02/09/2023] [Indexed: 04/16/2023] Open
Abstract
A quantitative systems pharmacology (QSP) model of the pathogenesis and treatment of SARS-CoV-2 infection can streamline and accelerate the development of novel medicines to treat COVID-19. Simulation of clinical trials allows in silico exploration of the uncertainties of clinical trial design and can rapidly inform their protocols. We previously published a preliminary model of the immune response to SARS-CoV-2 infection. To further our understanding of COVID-19 and treatment, we significantly updated the model by matching a curated dataset spanning viral load and immune responses in plasma and lung. We identified a population of parameter sets to generate heterogeneity in pathophysiology and treatment and tested this model against published reports from interventional SARS-CoV-2 targeting mAb and antiviral trials. Upon generation and selection of a virtual population, we match both the placebo and treated responses in viral load in these trials. We extended the model to predict the rate of hospitalization or death within a population. Via comparison of the in silico predictions with clinical data, we hypothesize that the immune response to virus is log-linear over a wide range of viral load. To validate this approach, we show the model matches a published subgroup analysis, sorted by baseline viral load, of patients treated with neutralizing Abs. By simulating intervention at different time points post infection, the model predicts efficacy is not sensitive to interventions within five days of symptom onset, but efficacy is dramatically reduced if more than five days pass post symptom onset prior to treatment.
Collapse
Affiliation(s)
- Rohit Rao
- Early Clinical Development, Pfizer Worldwide Research, Development and Medical, Cambridge, MA, USA.
| | - Cynthia J Musante
- Early Clinical Development, Pfizer Worldwide Research, Development and Medical, Cambridge, MA, USA
| | - Richard Allen
- Early Clinical Development, Pfizer Worldwide Research, Development and Medical, Cambridge, MA, USA
| |
Collapse
|
225
|
Ahmad J, Saudagar AKJ, Malik KM, Khan MB, AlTameem A, Alkhathami M, Hasanat MHA. Prognosis Prediction in COVID-19 Patients through Deep Feature Space Reasoning. Diagnostics (Basel) 2023; 13:diagnostics13081387. [PMID: 37189488 DOI: 10.3390/diagnostics13081387] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Revised: 03/05/2023] [Accepted: 03/17/2023] [Indexed: 05/17/2023] Open
Abstract
The COVID-19 pandemic has presented a unique challenge for physicians worldwide, as they grapple with limited data and uncertainty in diagnosing and predicting disease outcomes. In such dire circumstances, the need for innovative methods that can aid in making informed decisions with limited data is more critical than ever before. To allow prediction with limited COVID-19 data as a case study, we present a complete framework for progression and prognosis prediction in chest X-rays (CXR) through reasoning in a COVID-specific deep feature space. The proposed approach relies on a pre-trained deep learning model that has been fine-tuned specifically for COVID-19 CXRs to identify infection-sensitive features from chest radiographs. Using a neuronal attention-based mechanism, the proposed method determines dominant neural activations that lead to a feature subspace where neurons are more sensitive to COVID-related abnormalities. This process allows the input CXRs to be projected into a high-dimensional feature space where age and clinical attributes like comorbidities are associated with each CXR. The proposed method can accurately retrieve relevant cases from electronic health records (EHRs) using visual similarity, age group, and comorbidity similarities. These cases are then analyzed to gather evidence for reasoning, including diagnosis and treatment. By using a two-stage reasoning process based on the Dempster-Shafer theory of evidence, the proposed method can accurately predict the severity, progression, and prognosis of a COVID-19 patient when sufficient evidence is available. Experimental results on two large datasets show that the proposed method achieves 88% precision, 79% recall, and 83.7% F-score on the test sets.
Collapse
Affiliation(s)
- Jamil Ahmad
- Department of Computer Science, Islamia College Peshawar, Peshawar 25120, Pakistan
| | | | - Khalid Mahmood Malik
- Department of Computer Science and Engineering, Oakland University, Rochester, MI 48309, USA
| | - Muhammad Badruddin Khan
- Information Systems Department, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh 11432, Saudi Arabia
| | - Abdullah AlTameem
- Information Systems Department, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh 11432, Saudi Arabia
| | - Mohammed Alkhathami
- Information Systems Department, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh 11432, Saudi Arabia
| | | |
Collapse
|
226
|
Zahra A, Luijken K, Abbink EJ, van den Berg JM, Blom MT, Elders P, Festen J, Gussekloo J, Joling KJ, Melis R, Mooijaart S, Peters JB, Polinder-Bos HA, van Raaij BFM, Smorenberg A, la Roi-Teeuw HM, Moons KGM, van Smeden M. A study protocol of external validation of eight COVID-19 prognostic models for predicting mortality risk in older populations in a hospital, primary care, and nursing home setting. Diagn Progn Res 2023; 7:8. [PMID: 37013651 PMCID: PMC10069944 DOI: 10.1186/s41512-023-00144-2] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Accepted: 01/27/2023] [Indexed: 04/05/2023] Open
Abstract
BACKGROUND The COVID-19 pandemic has a large impact worldwide and is known to particularly affect the older population. This paper outlines the protocol for external validation of prognostic models predicting mortality risk after presentation with COVID-19 in the older population. These prognostic models were originally developed in an adult population and will be validated in an older population (≥ 70 years of age) in three healthcare settings: the hospital setting, the primary care setting, and the nursing home setting. METHODS Based on a living systematic review of COVID-19 prediction models, we identified eight prognostic models predicting the risk of mortality in adults with a COVID-19 infection (five COVID-19 specific models: GAL-COVID-19 mortality, 4C Mortality Score, NEWS2 + model, Xie model, and Wang clinical model and three pre-existing prognostic scores: APACHE-II, CURB65, SOFA). These eight models will be validated in six different cohorts of the Dutch older population (three hospital cohorts, two primary care cohorts, and a nursing home cohort). All prognostic models will be validated in a hospital setting while the GAL-COVID-19 mortality model will be validated in hospital, primary care, and nursing home settings. The study will include individuals ≥ 70 years of age with a highly suspected or PCR-confirmed COVID-19 infection from March 2020 to December 2020 (and up to December 2021 in a sensitivity analysis). The predictive performance will be evaluated in terms of discrimination, calibration, and decision curves for each of the prognostic models in each cohort individually. For prognostic models with indications of miscalibration, an intercept update will be performed after which predictive performance will be re-evaluated. DISCUSSION Insight into the performance of existing prognostic models in one of the most vulnerable populations clarifies the extent to which tailoring of COVID-19 prognostic models is needed when models are applied to the older population. Such insight will be important for possible future waves of the COVID-19 pandemic or future pandemics.
Collapse
Affiliation(s)
- Anum Zahra
- Julius Center for Health Sciences and Primary Care, University Medical Center, Utrecht University, Utrecht, the Netherlands.
| | - Kim Luijken
- Julius Center for Health Sciences and Primary Care, University Medical Center, Utrecht University, Utrecht, the Netherlands
| | - Evertine J Abbink
- Department of Internal Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Jesse M van den Berg
- Department of General Practice, Amsterdam UMC Location Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
- Amsterdam Public Health, Health Behaviors & Chronic Diseases, Amsterdam, the Netherlands
- PHARMO Institute for Drug Outcomes Research, Utrecht, the Netherlands
| | - Marieke T Blom
- Department of General Practice, Amsterdam UMC Location Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
- Amsterdam Public Health, Health Behaviors & Chronic Diseases, Amsterdam, the Netherlands
| | - Petra Elders
- Department of General Practice, Amsterdam Public Health Research Institute, Amsterdam UMC, Amsterdam, the Netherlands
| | | | - Jacobijn Gussekloo
- Department of Public Health and Primary Care & Department of Internal Medicine, Section of Gerontology and Geriatrics, Leiden University Medical Center, Leiden, the Netherlands
| | - Karlijn J Joling
- Department of Medicine for Older People, Amsterdam UMC, Location Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
- Amsterdam Public Health, Aging & Later Life, Amsterdam, the Netherlands
| | - René Melis
- Department of Geriatric Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Simon Mooijaart
- Department of Internal Medicine, Leiden University Medical Center, Leiden, the Netherlands
| | - Jeannette B Peters
- Department of Pulmonary Diseases, Radboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Harmke A Polinder-Bos
- Department of Internal Medicine, Section of Geriatric Medicine, Erasmus MC, University Medical Center Rotterdam, Rotterdam, the Netherlands
| | - Bas F M van Raaij
- Department of Internal Medicine, Leiden University Medical Center, Leiden, the Netherlands
| | - Annemieke Smorenberg
- Department of Internal Medicine, Section of Geriatric Medicine, Amsterdam UMC, Amsterdam, the Netherlands
| | - Hannah M la Roi-Teeuw
- Julius Center for Health Sciences and Primary Care, University Medical Center, Utrecht University, Utrecht, the Netherlands
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, University Medical Center, Utrecht University, Utrecht, the Netherlands
| | - Maarten van Smeden
- Julius Center for Health Sciences and Primary Care, University Medical Center, Utrecht University, Utrecht, the Netherlands
| |
Collapse
|
227
|
Searle T, Ibrahim Z, Teo J, Dobson RJB. Discharge summary hospital course summarisation of in patient Electronic Health Record text with clinical concept guided deep pre-trained Transformer models. J Biomed Inform 2023; 141:104358. [PMID: 37023846 DOI: 10.1016/j.jbi.2023.104358] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Revised: 03/29/2023] [Accepted: 04/02/2023] [Indexed: 04/08/2023]
Abstract
Brief Hospital Course (BHC) summaries are succinct summaries of an entire hospital encounter, embedded within discharge summaries, written by senior clinicians responsible for the overall care of a patient. Methods to automatically produce summaries from inpatient documentation would be invaluable in reducing clinician manual burden of summarising documents under high time-pressure to admit and discharge patients. Automatically producing these summaries from the inpatient course, is a complex, multi-document summarisation task, as source notes are written from various perspectives (e.g. nursing, doctor, radiology), during the course of the hospitalisation. We demonstrate a range of methods for BHC summarisation demonstrating the performance of deep learning summarisation models across extractive and abstractive summarisation scenarios. We also test a novel ensemble extractive and abstractive summarisation model that incorporates a medical concept ontology (SNOMED) as a clinical guidance signal and shows superior performance in 2 real-world clinical data sets.
Collapse
Affiliation(s)
- Thomas Searle
- Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK.
| | - Zina Ibrahim
- Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - James Teo
- King's College Hospital NHS Foundation Trust, London, UK
| | - Richard J B Dobson
- Department of Biostatistics and Health Informatics, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK; Institute of Health Informatics, University College London, London, UK
| |
Collapse
|
228
|
Odagaki T. New compartment model for COVID-19. Sci Rep 2023; 13:5409. [PMID: 37012332 PMCID: PMC10068699 DOI: 10.1038/s41598-023-32159-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/27/2022] [Accepted: 03/23/2023] [Indexed: 04/05/2023] Open
Abstract
The SIR or susceptible-infected-recovered model is the standard compartment model for understanding epidemics and has been used all over the world for COVID-19. While the SIR model assumes that infected patients are identical to symptomatic and infectious patients, it is now known that in COVID-19 pre-symptomatic patients are infectious and there are significant number of asymptomatic patients who are infectious. In this paper, population is separated into five compartments for COVID-19; susceptible individuals (S), pre-symptomatic patients (P), asymptomatic patients (A), quarantined patients (Q) and recovered and/or dead patients (R). The time evolution of population in each compartment is described by a set of ordinary differential equations. Numerical solution to the set of differential equations shows that quarantining pre-symptomatic and asymptomatic patients is effective in controlling the pandemic.
Collapse
Affiliation(s)
- Takashi Odagaki
- Kyushu University, Fukuoka, 819-0395, Japan.
- Research Institute for Science Education, Inc., Kyoto, 603-8346, Japan.
| |
Collapse
|
229
|
Su N, Donders MCHCM, Ho JPTF, Vespasiano V, de Lange J, Loos BG. Development and external validation of prediction models for critical outcomes of unvaccinated COVID-19 patients based on demographics, medical conditions and dental status. Heliyon 2023; 9:e15283. [PMID: 37064437 PMCID: PMC10084632 DOI: 10.1016/j.heliyon.2023.e15283] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Revised: 03/18/2023] [Accepted: 03/31/2023] [Indexed: 04/18/2023] Open
Abstract
Background Multiple prediction models were developed for critical outcomes of COVID-19. However, prediction models using predictors which can be easily obtained in clinical practice and on dental status are scarce. Aim The study aimed to develop and externally validate prediction models for critical outcomes of COVID-19 for unvaccinated adult patients in hospital settings based on demographics, medical conditions, and dental status. Methods A total of 285 and 352 patients from two hospitals in the Netherlands were retrospectively included as derivation and validation cohorts. Demographics, medical conditions, and dental status were considered potential predictors. The critical outcomes (death and ICU admission) were considered endpoints. Logistic regression analyses were used to develop two models: for death alone and for critical outcomes. The performance and clinical values of the models were determined in both cohorts. Results Age, number of teeth, chronic kidney disease, hypertension, diabetes, and chronic obstructive pulmonary diseases were the significant independent predictors. The models showed good to excellent calibration with observed: expected (O:E) ratios of 0.98 (95%CI: 0.76 to 1.25) and 1.00 (95%CI: 0.80 to 1.24), and discrimination with shrunken area under the curve (AUC) values of 0.85 and 0.79, based on the derivation cohort. In the validation cohort, the models showed good to excellent discrimination with AUC values of 0.85 (95%CI: 0.80 to 0.90) and 0.78 (95%CI: 0.73 to 0.83), but an overestimation in calibration with O:E ratios of 0.65 (95%CI: 0.49 to 0.85) and 0.67 (95%CI: 0.52 to 0.84). Conclusion The performance of the models was acceptable in both derivation and validation cohorts. Number of teeth was an additive important predictor of critical outcomes of COVID-19. It is an easy-to-apply tool in hospitals for risk stratification of COVID-19 prognosis.
Collapse
Affiliation(s)
- Naichuan Su
- Department of Oral Public Health, Academic Centre for Dentistry Amsterdam (ACTA), University of Amsterdam and Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| | - Marie-Chris H C M Donders
- Department of Oral and Maxillofacial Surgery, Amsterdam UMC, Academic Centre for Dentistry Amsterdam (ACTA), University of Amsterdam, Amsterdam, the Netherlands
- Department of Oral and Maxillofacial Surgery, Isala Zwolle, Zwolle, the Netherlands
| | - Jean-Pierre T F Ho
- Department of Oral and Maxillofacial Surgery, Amsterdam UMC, Academic Centre for Dentistry Amsterdam (ACTA), University of Amsterdam, Amsterdam, the Netherlands
- Department of Oral and Maxillofacial Surgery, Northwest Clinics, Alkmaar, the Netherlands
| | - Valeria Vespasiano
- Department of Oral and Maxillofacial Surgery, Amsterdam UMC, Academic Centre for Dentistry Amsterdam (ACTA), University of Amsterdam, Amsterdam, the Netherlands
| | - Jan de Lange
- Department of Oral and Maxillofacial Surgery, Amsterdam UMC, Academic Centre for Dentistry Amsterdam (ACTA), University of Amsterdam, Amsterdam, the Netherlands
- Department of Oral and Maxillofacial Surgery, Isala Zwolle, Zwolle, the Netherlands
| | - Bruno G Loos
- Department of Periodontology, Academic Centre for Dentistry Amsterdam (ACTA), University of Amsterdam and Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| |
Collapse
|
230
|
Liu TL, Chou SH, Murphy S, Kowalkowski M, Taylor YJ, Hole C, Sitammagari K, Priem JS, McWilliams A. Evaluating Racial/Ethnic Differences in Care Escalation Among COVID-19 Patients in a Home-Based Hospital. J Racial Ethn Health Disparities 2023; 10:817-825. [PMID: 35257312 PMCID: PMC8900643 DOI: 10.1007/s40615-022-01270-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2021] [Revised: 02/16/2022] [Accepted: 02/19/2022] [Indexed: 11/25/2022]
Abstract
The novel coronavirus disease 2019 (COVID-19) has infected over 414 million people worldwide with 5.8 million deaths, as of February 2022. Telemedicine-based interventions to expand healthcare systems' capacity and reduce infection risk have rapidly increased during the pandemic, despite concerns regarding equitable access. Atrium Health Hospital at Home (AH-HaH) is a home-based program that provides advanced, hospital-level medical care and monitoring for patients who would otherwise be hospitalized in a traditional setting. Our retrospective cohort study of positive COVID-19 patients who were admitted to AH-HaH aims to investigate whether the rate of care escalation from AH-HaH to traditional hospitalization differed based on patients' racial/ethnic backgrounds. Logistic regression was used to examine the association between care escalation within 14 days from index AH-HaH admission and race/ethnicity. We found approximately one in five patients receiving care for COVID-19 in AH-HaH required care escalation within 14 days. Odds of care escalation were not significantly different for Hispanic or non-Hispanic Blacks compared to non-Hispanic Whites. However, secondary analyses showed that both Hispanic and non-Hispanic Black patients were younger and with fewer comorbidities than non-Hispanic Whites. The study highlights the need for new care models to vigilantly monitor for disparities, so that timely and tailored adaptations can be implemented for vulnerable populations.
Collapse
Affiliation(s)
- Tsai-Ling Liu
- Center for Outcomes Research and Evaluation (CORE), Atrium Health, 1300 Scott Ave, Charlotte, NC, 28204, USA.
| | - Shih-Hsiung Chou
- Center for Outcomes Research and Evaluation (CORE), Atrium Health, 1300 Scott Ave, Charlotte, NC, 28204, USA
| | - Stephanie Murphy
- Division of Hospital Medicine, Department of Internal Medicine, Atrium Health, Charlotte, NC, USA
| | - Marc Kowalkowski
- Center for Outcomes Research and Evaluation (CORE), Atrium Health, 1300 Scott Ave, Charlotte, NC, 28204, USA
| | - Yhenneko J Taylor
- Center for Outcomes Research and Evaluation (CORE), Atrium Health, 1300 Scott Ave, Charlotte, NC, 28204, USA
| | - Colleen Hole
- Population Health, Atrium Health, Charlotte, NC, USA
| | - Kranthi Sitammagari
- Division of Hospital Medicine, Department of Internal Medicine, Atrium Health, Charlotte, NC, USA
| | - Jennifer S Priem
- Center for Outcomes Research and Evaluation (CORE), Atrium Health, 1300 Scott Ave, Charlotte, NC, 28204, USA
| | - Andrew McWilliams
- Center for Outcomes Research and Evaluation (CORE), Atrium Health, 1300 Scott Ave, Charlotte, NC, 28204, USA.,Division of Hospital Medicine, Department of Internal Medicine, Atrium Health, Charlotte, NC, USA
| |
Collapse
|
231
|
Buttia C, Llanaj E, Raeisi-Dehkordi H, Kastrati L, Amiri M, Meçani R, Taneri PE, Ochoa SAG, Raguindin PF, Wehrli F, Khatami F, Espínola OP, Rojas LZ, de Mortanges AP, Macharia-Nimietz EF, Alijla F, Minder B, Leichtle AB, Lüthi N, Ehrhard S, Que YA, Fernandes LK, Hautz W, Muka T. Prognostic models in COVID-19 infection that predict severity: a systematic review. Eur J Epidemiol 2023; 38:355-372. [PMID: 36840867 PMCID: PMC9958330 DOI: 10.1007/s10654-023-00973-x] [Citation(s) in RCA: 13] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Accepted: 01/28/2023] [Indexed: 02/26/2023]
Abstract
Current evidence on COVID-19 prognostic models is inconsistent and clinical applicability remains controversial. We performed a systematic review to summarize and critically appraise the available studies that have developed, assessed and/or validated prognostic models of COVID-19 predicting health outcomes. We searched six bibliographic databases to identify published articles that investigated univariable and multivariable prognostic models predicting adverse outcomes in adult COVID-19 patients, including intensive care unit (ICU) admission, intubation, high-flow nasal therapy (HFNT), extracorporeal membrane oxygenation (ECMO) and mortality. We identified and assessed 314 eligible articles from more than 40 countries, with 152 of these studies presenting mortality, 66 progression to severe or critical illness, 35 mortality and ICU admission combined, 17 ICU admission only, while the remaining 44 studies reported prediction models for mechanical ventilation (MV) or a combination of multiple outcomes. The sample size of included studies varied from 11 to 7,704,171 participants, with a mean age ranging from 18 to 93 years. There were 353 prognostic models investigated, with area under the curve (AUC) ranging from 0.44 to 0.99. A great proportion of studies (61.5%, 193 out of 314) performed internal or external validation or replication. In 312 (99.4%) studies, prognostic models were reported to be at high risk of bias due to uncertainties and challenges surrounding methodological rigor, sampling, handling of missing data, failure to deal with overfitting and heterogeneous definitions of COVID-19 and severity outcomes. While several clinical prognostic models for COVID-19 have been described in the literature, they are limited in generalizability and/or applicability due to deficiencies in addressing fundamental statistical and methodological concerns. Future large, multi-centric and well-designed prognostic prospective studies are needed to clarify remaining uncertainties.
Collapse
Affiliation(s)
- Chepkoech Buttia
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Emergency Department, Inselspital, Bern University Hospital, University of Bern, Freiburgstrasse 16C, 3010 Bern, Switzerland
- Epistudia, Bern, Switzerland
| | - Erand Llanaj
- Department of Molecular Epidemiology, German Institute of Human Nutrition Potsdam-Rehbrücke, Nuthetal, Germany
- ELKH-DE Public Health Research Group of the Hungarian Academy of Sciences, Department of Public Health and Epidemiology, Faculty of Medicine, University of Debrecen, Debrecen, Hungary
- Epistudia, Bern, Switzerland
- German Center for Diabetes Research (DZD), München-Neuherberg, Germany
| | - Hamidreza Raeisi-Dehkordi
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Lum Kastrati
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Graduate School for Health Sciences, University of Bern, Bern, Switzerland
- Department of Diabetes, Endocrinology, Nutritional Medicine and Metabolism, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| | - Mojgan Amiri
- Department of Epidemiology, Erasmus MC University Medical Center, Rotterdam, The Netherlands
| | - Renald Meçani
- Department of Pediatrics, “Mother Teresa” University Hospital Center, Tirana, University of Medicine, Tirana, Albania
- Division of Endocrinology and Diabetology, Department of Internal Medicine, Medical University of Graz, Graz, Austria
| | - Petek Eylul Taneri
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- HRB-Trials Methodology Research Network College of Medicine, Nursing and Health Sciences University of Galway, Galway, Ireland
| | | | - Peter Francis Raguindin
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Swiss Paraplegic Research, Nottwil, Switzerland
- Faculty of Health Sciences, University of Lucerne, Lucerne, Switzerland
| | - Faina Wehrli
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
| | - Farnaz Khatami
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Graduate School for Health Sciences, University of Bern, Bern, Switzerland
- Department of Community Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - Octavio Pano Espínola
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Department of Preventive Medicine and Public Health, University of Navarre, Pamplona, Spain
- Navarra Institute for Health Research, IdiSNA, Pamplona, Spain
| | - Lyda Z. Rojas
- Research Group and Development of Nursing Knowledge (GIDCEN-FCV), Research Center, Cardiovascular Foundation of Colombia, Floridablanca, Santander, Colombia
| | | | | | - Fadi Alijla
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Graduate School for Health Sciences, University of Bern, Bern, Switzerland
| | - Beatrice Minder
- Public Health and Primary Care Library, University Library of Bern, University of Bern, Bern, Switzerland
| | - Alexander B. Leichtle
- University Institute of Clinical Chemistry, Inselspital, Bern University Hospital, and Center for Artificial Intelligence in Medicine (CAIM), University of Bern, Bern, Switzerland
| | - Nora Lüthi
- Emergency Department, Inselspital, Bern University Hospital, University of Bern, Freiburgstrasse 16C, 3010 Bern, Switzerland
| | - Simone Ehrhard
- Emergency Department, Inselspital, Bern University Hospital, University of Bern, Freiburgstrasse 16C, 3010 Bern, Switzerland
| | - Yok-Ai Que
- Department of Intensive Care Medicine, Inselspital, Bern University Hospital, University of Bern, Bern, Switzerland
| | - Laurenz Kopp Fernandes
- Deutsches Herzzentrum Berlin (DHZB), Berlin, Germany
- Charité Universitätsmedizin Berlin, Berlin, Germany
| | - Wolf Hautz
- Emergency Department, Inselspital, Bern University Hospital, University of Bern, Freiburgstrasse 16C, 3010 Bern, Switzerland
| | - Taulant Muka
- Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
- Epistudia, Bern, Switzerland
| |
Collapse
|
232
|
Gu Y, Wu W, Kong C, Luo Q, Ran L, Tan X, Zhang Q. Development of Prognostic Prediction Model to Estimate Mortality for Frail Oldest Old: Prospective Cohort Study. J Gerontol A Biol Sci Med Sci 2023; 78:711-717. [PMID: 36574506 PMCID: PMC10061936 DOI: 10.1093/gerona/glac256] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Indexed: 12/29/2022] Open
Abstract
BACKGROUND This study was performed to derive and validate a prognostic prediction model for individualized estimation of mortality risk among the frail oldest old (aged 80 years or older). METHODS This analysis was based on the prospective open cohort study from the Chinese Longevity and Health Longitudinal Survey. A total of 14 118 frail oldest old were included from the 2002 wave to 2014 waves; the study outcome was all-cause mortality. Available predictors included frailty, demographics, and social factors. Cox models were used to estimate the coefficients of the predictors and least absolute shrinkage and selection operator was used for selecting predictors. Model performance was measured by discrimination and calibration with internal validation by bootstrapping. We also developed a nomogram to visualize and predict the 3-year mortality risk based on the obtained prognostic prediction model. RESULTS During the 16-years follow-up, 10 410 (76.42%) deaths were identified. The final model comprises the following factors: frailty, age, sex, race, birthplace, education, occupation, marital status, residence, economic condition, number of children, and the question "who do you ask for help first when in trouble." The model has valid predictive ability as measured and validated by Harrell's C statistic (0.602) and calibration plots. CONCLUSIONS This study provides a basic prognostic prediction model to quantify absolute mortality risk for the frail oldest old. Future studies are needed, firstly, to update, adjust, and perform external validation of the present model by using phenotypic frailty, and secondly, to add biomarkers, environmental, and psychological factors to the prediction model.
Collapse
Affiliation(s)
- Yaohua Gu
- School of Nursing, Wuhan University, Wuhan, China
| | - Wenwen Wu
- School of Public Health and Management, Hubei University of Medicine, Hubei, China
| | - Chan Kong
- Tongji Hospital, Tongji Medical College of Huazhong University of Science & Technology, Wuhan, China
| | - Qiaoqian Luo
- Department of General Medicine, Zhongnan Hospital, Wuhan University, Wuhan, China
| | - Li Ran
- School of Public Health, Wuhan University, Wuhan, China
| | - Xiaodong Tan
- School of Public Health, Wuhan University, Wuhan, China
| | - Qing Zhang
- School of Nursing, Wuhan University, Wuhan, China
| |
Collapse
|
233
|
Karpov OE, Pitsik EN, Kurkin SA, Maksimenko VA, Gusev AV, Shusharina NN, Hramov AE. Analysis of Publication Activity and Research Trends in the Field of AI Medical Applications: Network Approach. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2023; 20:5335. [PMID: 37047950 PMCID: PMC10094658 DOI: 10.3390/ijerph20075335] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/14/2023] [Revised: 03/17/2023] [Accepted: 03/22/2023] [Indexed: 06/19/2023]
Abstract
Artificial intelligence (AI) has revolutionized numerous industries, including medicine. In recent years, the integration of AI into medical practices has shown great promise in enhancing the accuracy and efficiency of diagnosing diseases, predicting patient outcomes, and personalizing treatment plans. This paper aims at the exploration of the AI-based medicine research using network approach and analysis of existing trends based on PubMed. Our findings are based on the results of PubMed search queries and analysis of the number of papers obtained by the different search queries. Our goal is to explore how are the AI-based methods used in healthcare research, which approaches and techniques are the most popular, and to discuss the potential reasoning behind the obtained results. Using analysis of the co-occurrence network constructed using VOSviewer software, we detected the main clusters of interest in AI-based healthcare research. Then, we proceeded with the thorough analysis of publication activity in various categories of medical AI research, including research on different AI-based methods applied to different types of medical data. We analyzed the results of query processing in the PubMed database over the past 5 years obtained via a specifically designed strategy for generating search queries based on the thorough selection of keywords from different categories of interest. We provide a comprehensive analysis of existing applications of AI-based methods to medical data of different modalities, including the context of various medical fields and specific diseases that carry the greatest danger to the human population.
Collapse
Affiliation(s)
- Oleg E. Karpov
- National Medical and Surgical Center Named after N. I. Pirogov, Ministry of Healthcare of the Russian Federation, 105203 Moscow, Russia
| | - Elena N. Pitsik
- Baltic Center for Neurotechnology and Artificial Intelligence, Immanuel Kant Baltic Federal University, 236041 Kaliningrad, Russia; (E.N.P.); (S.A.K.); (V.A.M.); (N.N.S.)
| | - Semen A. Kurkin
- Baltic Center for Neurotechnology and Artificial Intelligence, Immanuel Kant Baltic Federal University, 236041 Kaliningrad, Russia; (E.N.P.); (S.A.K.); (V.A.M.); (N.N.S.)
| | - Vladimir A. Maksimenko
- Baltic Center for Neurotechnology and Artificial Intelligence, Immanuel Kant Baltic Federal University, 236041 Kaliningrad, Russia; (E.N.P.); (S.A.K.); (V.A.M.); (N.N.S.)
| | - Alexander V. Gusev
- K-Skai LLC, 185031 Petrozavodsk, Russia
- Federal Research Institute for Health Organization and Informatics, 127254 Moscow, Russia
| | - Natali N. Shusharina
- Baltic Center for Neurotechnology and Artificial Intelligence, Immanuel Kant Baltic Federal University, 236041 Kaliningrad, Russia; (E.N.P.); (S.A.K.); (V.A.M.); (N.N.S.)
| | - Alexander E. Hramov
- Baltic Center for Neurotechnology and Artificial Intelligence, Immanuel Kant Baltic Federal University, 236041 Kaliningrad, Russia; (E.N.P.); (S.A.K.); (V.A.M.); (N.N.S.)
| |
Collapse
|
234
|
Tangirala S, Tierney BT, Patel CJ. Prioritization of COVID-19 risk factors in July 2020 and February 2021 in the UK. COMMUNICATIONS MEDICINE 2023; 3:45. [PMID: 36997659 PMCID: PMC10062272 DOI: 10.1038/s43856-023-00271-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2022] [Accepted: 03/07/2023] [Indexed: 04/04/2023] Open
Abstract
BACKGROUND Risk for COVID-19 positivity and hospitalization due to diverse environmental and sociodemographic factors may change as the pandemic progresses. METHODS We investigated the association of 360 exposures sampled before COVID-19 outcomes for participants in the UK Biobank, including 9268 and 38,837 non-overlapping participants, sampled at July 17, 2020 and February 2, 2021, respectively. The 360 exposures included clinical biomarkers (e.g., BMI), health indicators (e.g., doctor-diagnosed diabetes), and environmental/behavioral variables (e.g., air pollution) measured 10-14 years before the COVID-19 time periods. RESULTS Here we show, for example, "participant having son and/or daughter in household" was associated with an increase in incidence from 20% to 32% (risk difference of 12%) between timepoints. Furthermore, we find age to be increasingly associated with COVID-19 positivity over time from Risk Ratio [RR] (per 10-year age increase) of 0.81 to 0.6 (hospitalization RR from 1.18 to 2.63, respectively). CONCLUSIONS Our data-driven approach demonstrates that time of pandemic plays a role in identifying risk factors associated with positivity and hospitalization.
Collapse
Affiliation(s)
- Sivateja Tangirala
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
- Department of Statistics and Data Science, Cornell University, Ithaca, NY, USA
| | - Braden T Tierney
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Chirag J Patel
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
235
|
Poznyak A, Chairez I, Anyutin A. Differential Neural Networks Prediction Using Slow and Fast Hybrid Learning: Application to Prognosis of Infectionsand Deaths of COVID-19 Dynamics. Neural Process Lett 2023:1-17. [PMID: 37359130 PMCID: PMC10035488 DOI: 10.1007/s11063-023-11216-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/26/2023] [Indexed: 03/25/2023]
Abstract
This essay discusses a potential method for predicting the behavior of various physical processes and uses the COVID-19 outbreak to demonstrate its applicability. This study assumes that the current data set reflects the output of a dynamic system that is governed by a nonlinear ordinary differential equation. This dynamic system may be described by a Differential Neural Network (DNN) with time-varying weights matrix parameters. A new hybrid learning scheme based on the decomposition of the signal to be predicted. The decomposition considers the slow and fast components of the signal which is more natural to signals such as the ones corresponding to the number of infected and deceased patients who suffered of COVID 2019 sickness. The paper results demonstrate the recommended method offers competitive performance (70 days of COVID prediction) in comparison to similar studies.
Collapse
Affiliation(s)
- A. Poznyak
- CINVESTAV IPN, DCA, Cd. de Mexico, Mexico
| | - I. Chairez
- Tecnologico de Monterrey, Institute of Advanced Materials for Sustainable Manufacturing, Cd. de Guadalajara, Mexico
| | - A. Anyutin
- Institute of Radio Engineering and Electronics, Fryazino Branch, Ran, Fryazino, Russia
| |
Collapse
|
236
|
Aboumrad M, Zwain G, Smith J, Neupane N, Powell E, Dempsey B, Reyes C, Satram S, Young-Xu Y. Development and Validation of a Clinical Risk Score to Predict Hospitalization Within 30 Days of Coronavirus Disease 2019 Diagnosis. Mil Med 2023; 188:e833-e840. [PMID: 34611704 PMCID: PMC8522374 DOI: 10.1093/milmed/usab415] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2021] [Revised: 08/26/2021] [Accepted: 09/28/2021] [Indexed: 11/30/2022] Open
Abstract
INTRODUCTION Early identification of patients with coronavirus disease 2019 (COVID-19) who are at risk for hospitalization may help to mitigate disease burden by allowing healthcare systems to conduct sufficient resource and logistical planning in the event of case surges. We sought to develop and validate a clinical risk score that uses readily accessible information at testing to predict individualized 30-day hospitalization risk following COVID-19 diagnosis. METHODS We assembled a retrospective cohort of U.S. Veterans Health Administration patients (age ≥ 18 years) diagnosed with COVID-19 between March 1, 2020, and December 31, 2020. We screened patient characteristics using Least Absolute Shrinkage and Selection Operator logistic regression and constructed the risk score using characteristics identified as most predictive for hospitalization. Patients diagnosed before November 1, 2020, comprised the development cohort, while those diagnosed on or after November 1, 2020, comprised the validation cohort. We assessed risk score discrimination by calculating the area under the receiver operating characteristic (AUROC) curve and calibration using the Hosmer-Lemeshow (HL) goodness-of-fit test. This study was approved by the Veteran's Institutional Review Board of Northern New England at the White River Junction Veterans Affairs Medical Center (Reference no.:1473972-1). RESULTS The development and validation cohorts comprised 11,473 and 12,970 patients, of whom 4,465 (38.9%) and 3,669 (28.3%) were hospitalized, respectively. The independent predictors for hospitalization included in the risk score were increasing age, male sex, non-white race, Hispanic ethnicity, homelessness, nursing home/long-term care residence, unemployed or retired status, fever, fatigue, diarrhea, nausea, cough, diabetes, chronic kidney disease, hypertension, and chronic obstructive pulmonary disease. Model discrimination and calibration was good for the development (AUROC = 0.80; HL P-value = .05) and validation (AUROC = 0.80; HL P-value = .31) cohorts. CONCLUSIONS The prediction tool developed in this study demonstrated that it could identify patients with COVID-19 who are at risk for hospitalization. This could potentially inform clinicians and policymakers of patients who may benefit most from early treatment interventions and help healthcare systems anticipate capacity surges.
Collapse
Affiliation(s)
- Maya Aboumrad
- Clinical Epidemiology Program, White River Junction Veterans Affairs Medical Center, White River Junction, VT 05009, USA
| | - Gabrielle Zwain
- Clinical Epidemiology Program, White River Junction Veterans Affairs Medical Center, White River Junction, VT 05009, USA
| | - Jeremy Smith
- Clinical Epidemiology Program, White River Junction Veterans Affairs Medical Center, White River Junction, VT 05009, USA
| | - Nabin Neupane
- Clinical Epidemiology Program, White River Junction Veterans Affairs Medical Center, White River Junction, VT 05009, USA
| | - Ethan Powell
- Clinical Epidemiology Program, White River Junction Veterans Affairs Medical Center, White River Junction, VT 05009, USA
| | - Brendan Dempsey
- Clinical Epidemiology Program, White River Junction Veterans Affairs Medical Center, White River Junction, VT 05009, USA
| | - Carolina Reyes
- Division of Health Economics and Outcomes Research, VIR Biotechnology Inc., San Francisco, CA 94158, USA
| | - Sacha Satram
- Division of Health Economics and Outcomes Research, VIR Biotechnology Inc., San Francisco, CA 94158, USA
| | - Yinong Young-Xu
- Clinical Epidemiology Program, White River Junction Veterans Affairs Medical Center, White River Junction, VT 05009, USA
- Department of Psychiatry, Geisel School of Medicine at Dartmouth, Hanover, NH 03755, USA
| |
Collapse
|
237
|
Junior EPP, Normando P, Flores-Ortiz R, Afzal MU, Jamil MA, Bertolin SF, Oliveira VDA, Martufi V, de Sousa F, Bashir A, Burn E, Ichihara MY, Barreto ML, Salles TD, Prieto-Alhambra D, Hafeez H, Khalid S. Integrating real-world data from Brazil and Pakistan into the OMOP common data model and standardized health analytics framework to characterize COVID-19 in the Global South. J Am Med Inform Assoc 2023; 30:643-655. [PMID: 36264262 PMCID: PMC9619798 DOI: 10.1093/jamia/ocac180] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2022] [Revised: 08/16/2022] [Accepted: 09/29/2022] [Indexed: 11/13/2022] Open
Abstract
OBJECTIVES The aim of this work is to demonstrate the use of a standardized health informatics framework to generate reliable and reproducible real-world evidence from Latin America and South Asia towards characterizing coronavirus disease 2019 (COVID-19) in the Global South. MATERIALS AND METHODS Patient-level COVID-19 records collected in a patient self-reported notification system, hospital in-patient and out-patient records, and community diagnostic labs were harmonized to the Observational Medical Outcomes Partnership common data model and analyzed using a federated network analytics framework. Clinical characteristics of individuals tested for, diagnosed with or tested positive for, hospitalized with, admitted to intensive care unit with, or dying with COVID-19 were estimated. RESULTS Two COVID-19 databases covering 8.3 million people from Pakistan and 2.6 million people from Bahia, Brazil were analyzed. 109 504 (Pakistan) and 921 (Brazil) medical concepts were harmonized to Observational Medical Outcomes Partnership common data model. In total, 341 505 (4.1%) people in the Pakistan dataset and 1 312 832 (49.2%) people in the Brazilian dataset were tested for COVID-19 between January 1, 2020 and April 20, 2022, with a median [IQR] age of 36 [25, 76] and 38 (27, 50); 40.3% and 56.5% were female in Pakistan and Brazil, respectively. 1.2% percent individuals in the Pakistan dataset had Afghan ethnicity. In Brazil, 52.3% had mixed ethnicity. In agreement with international findings, COVID-19 outcomes were more severe in men, elderly, and those with underlying health conditions. CONCLUSIONS COVID-19 data from 2 large countries in the Global South were harmonized and analyzed using a standardized health informatics framework developed by an international community of health informaticians. This proof-of-concept study demonstrates a potential open science framework for global knowledge mobilization and clinical translation for timely response to healthcare needs in pandemics and beyond.
Collapse
Affiliation(s)
- Elzo Pereira Pinto Junior
- Center of Data and Knowledge Integration for Health (Cidacs), Fiocruz-Brazil, Parque Tecnológico da Edf, Tecnocentro, R. Mundo, Salvador, BA 41745-715, Brazil
| | - Priscilla Normando
- Center of Data and Knowledge Integration for Health (Cidacs), Fiocruz-Brazil, Parque Tecnológico da Edf, Tecnocentro, R. Mundo, Salvador, BA 41745-715, Brazil
| | - Renzo Flores-Ortiz
- Center of Data and Knowledge Integration for Health (Cidacs), Fiocruz-Brazil, Parque Tecnológico da Edf, Tecnocentro, R. Mundo, Salvador, BA 41745-715, Brazil
| | - Muhammad Usman Afzal
- Shaukat Khanum Memorial Cancer Hospital and Research Centre, Johar Town, Lahore, 54840, Pakistan
| | - Muhammad Asaad Jamil
- Shaukat Khanum Memorial Cancer Hospital and Research Centre, Johar Town, Lahore, 54840, Pakistan
| | - Sergio Fernandez Bertolin
- Fundació Institut, Universitari per a la recerca a l'Atenció Primària de Salut Jordi Gol i Gurina (IDIAPJGol), Barcelona, 587 08007, Spain
| | - Vinícius de Araújo Oliveira
- Center of Data and Knowledge Integration for Health (Cidacs), Fiocruz-Brazil, Parque Tecnológico da Edf, Tecnocentro, R. Mundo, Salvador, BA 41745-715, Brazil
| | - Valentina Martufi
- Center of Data and Knowledge Integration for Health (Cidacs), Fiocruz-Brazil, Parque Tecnológico da Edf, Tecnocentro, R. Mundo, Salvador, BA 41745-715, Brazil
| | - Fernanda de Sousa
- Center of Data and Knowledge Integration for Health (Cidacs), Fiocruz-Brazil, Parque Tecnológico da Edf, Tecnocentro, R. Mundo, Salvador, BA 41745-715, Brazil
| | - Amir Bashir
- Shaukat Khanum Memorial Cancer Hospital and Research Centre, Johar Town, Lahore, 54840, Pakistan
| | - Edward Burn
- Centre for Statistics in Medicine, Botnar Research Centre, University of Oxford, Oxford, OX3 7LD, United Kingdom
| | - Maria Yury Ichihara
- Center of Data and Knowledge Integration for Health (Cidacs), Fiocruz-Brazil, Parque Tecnológico da Edf, Tecnocentro, R. Mundo, Salvador, BA 41745-715, Brazil
| | - Maurício L Barreto
- Center of Data and Knowledge Integration for Health (Cidacs), Fiocruz-Brazil, Parque Tecnológico da Edf, Tecnocentro, R. Mundo, Salvador, BA 41745-715, Brazil
| | - Talita Duarte Salles
- Fundació Institut, Universitari per a la recerca a l'Atenció Primària de Salut Jordi Gol i Gurina (IDIAPJGol), Barcelona, 587 08007, Spain
| | - Daniel Prieto-Alhambra
- Centre for Statistics in Medicine, Botnar Research Centre, University of Oxford, Oxford, OX3 7LD, United Kingdom
| | - Haroon Hafeez
- Shaukat Khanum Memorial Cancer Hospital and Research Centre, Johar Town, Lahore, 54840, Pakistan
| | - Sara Khalid
- Centre for Statistics in Medicine, Botnar Research Centre, University of Oxford, Oxford, OX3 7LD, United Kingdom
| |
Collapse
|
238
|
Tuncer G, Geyiktepe-Guclu C, Surme S, Canel-Karakus E, Erdogan H, Bayramlar OF, Belge C, Karahasanoglu R, Copur B, Yazla M, Zerdali E, Nakir IY, Yildirim N, Kar B, Bozkurt M, Karanalbant K, Atasoy B, Takak H, Simsek-Yavuz S, Turkay R, M Sonmez M, Sengoz G, Pehlivanoglu F. Long-term effects of COVID-19 on lungs and the clinical relevance: a 6-month prospective cohort study. Future Microbiol 2023; 18:185-198. [PMID: 36916475 DOI: 10.2217/fmb-2022-0121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/16/2023] Open
Abstract
Background: We aimed to explore the prevalence of prolonged symptoms, pulmonary impairments and residual disease on chest tomography (CT) in COVID-19 patients at 6 months after acute illness. Methods: In this prospective, single-center study, hospitalized patients with radiologically and laboratory-confirmed COVID-19 were included. Results: A high proportion of the 116 patients reported persistent symptoms (n = 54; 46.6%). On follow-up CT, 33 patients (28.4%) demonstrated residual disease. Multivariate analyses revealed that only neutrophil-to-lymphocyte ratio was an independent predictor for residual disease. Conclusion: Hospitalized patients with mild/moderate COVID-19 still had persistent symptoms and were prone to develop long-term pulmonary sequelae on chest CT. However, it did not have a significant effect on long-term pulmonary functions.
Collapse
Affiliation(s)
- Gulsah Tuncer
- Department of Infectious Diseases & Clinical Microbiology, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Ceyda Geyiktepe-Guclu
- Department of Infectious Diseases & Clinical Microbiology, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Serkan Surme
- Department of Infectious Diseases & Clinical Microbiology, Haseki Training & Research Hospital, Istanbul, 34096, Turkey.,Department of Medical Microbiology, Institute of Graduate Studies, Istanbul University-Cerrahpasa, Istanbul, 34098, Turkey
| | - Evren Canel-Karakus
- Department of Pulmonary Medicine, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Hatice Erdogan
- Department of Microbiology & Clinical Microbiology, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Osman F Bayramlar
- Department of Public Health, Bakirkoy District Health Directorate, Istanbul, 34140, Turkey
| | - Cansu Belge
- Department of Radiology, Health Sciences University, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Ridvan Karahasanoglu
- Department of Radiology, Health Sciences University, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Betul Copur
- Department of Infectious Diseases & Clinical Microbiology, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Meltem Yazla
- Department of Infectious Diseases & Clinical Microbiology, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Esra Zerdali
- Department of Infectious Diseases & Clinical Microbiology, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Inci Y Nakir
- Department of Infectious Diseases & Clinical Microbiology, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Nihal Yildirim
- Department of Pulmonary Medicine, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Bedriye Kar
- Department of Pulmonary Medicine, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Mediha Bozkurt
- Department of Infectious Diseases & Clinical Microbiology, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Kubra Karanalbant
- Department of Infectious Diseases & Clinical Microbiology, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Burcu Atasoy
- Department of Infectious Diseases & Clinical Microbiology, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Hindirin Takak
- Department of Infectious Diseases & Clinical Microbiology, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Serap Simsek-Yavuz
- Department of Infectious Diseases & Clinical Microbiology, Istanbul Faculty of Medicine, Istanbul University, Istanbul, 34093, Turkey
| | - Rustu Turkay
- Department of Radiology, Health Sciences University, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Mehmet M Sonmez
- Department of Orthopedic Surgery & Traumatology, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Gonul Sengoz
- Department of Infectious Diseases & Clinical Microbiology, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| | - Filiz Pehlivanoglu
- Department of Infectious Diseases & Clinical Microbiology, Haseki Training & Research Hospital, Istanbul, 34096, Turkey
| |
Collapse
|
239
|
Kwok SWH, Wang G, Sohel F, Kashani KB, Zhu Y, Wang Z, Antpack E, Khandelwal K, Pagali SR, Nanda S, Abdalrhim AD, Sharma UM, Bhagra S, Dugani S, Takahashi PY, Murad MH, Yousufuddin M. An artificial intelligence approach for predicting death or organ failure after hospitalization for COVID-19: development of a novel risk prediction tool and comparisons with ISARIC-4C, CURB-65, qSOFA, and MEWS scoring systems. Respir Res 2023; 24:79. [PMID: 36915107 PMCID: PMC10010216 DOI: 10.1186/s12931-023-02386-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2022] [Accepted: 03/07/2023] [Indexed: 03/14/2023] Open
Abstract
BACKGROUND We applied machine learning (ML) algorithms to generate a risk prediction tool [Collaboration for Risk Evaluation in COVID-19 (CORE-COVID-19)] for predicting the composite of 30-day endotracheal intubation, intravenous administration of vasopressors, or death after COVID-19 hospitalization and compared it with the existing risk scores. METHODS This is a retrospective study of adults hospitalized with COVID-19 from March 2020 to February 2021. Patients, each with 92 variables, and one composite outcome underwent feature selection process to identify the most predictive variables. Selected variables were modeled to build four ML algorithms (artificial neural network, support vector machine, gradient boosting machine, and Logistic regression) and an ensemble model to generate a CORE-COVID-19 model to predict the composite outcome and compared with existing risk prediction scores. The net benefit for clinical use of each model was assessed by decision curve analysis. RESULTS Of 1796 patients, 278 (15%) patients reached primary outcome. Six most predictive features were identified. Four ML algorithms achieved comparable discrimination (P > 0.827) with c-statistics ranged 0.849-0.856, calibration slopes 0.911-1.173, and Hosmer-Lemeshow P > 0.141 in validation dataset. These 6-variable fitted CORE-COVID-19 model revealed a c-statistic of 0.880, which was significantly (P < 0.04) higher than ISARIC-4C (0.751), CURB-65 (0.735), qSOFA (0.676), and MEWS (0.674) for outcome prediction. The net benefit of the CORE-COVID-19 model was greater than that of the existing risk scores. CONCLUSION The CORE-COVID-19 model accurately assigned 88% of patients who potentially progressed to 30-day composite events and revealed improved performance over existing risk scores, indicating its potential utility in clinical practice.
Collapse
Affiliation(s)
| | - Guanjin Wang
- Department of Information Technology, Murdoch University, Murdoch, Australia
| | - Ferdous Sohel
- Department of Information Technology, Murdoch University, Murdoch, Australia
| | | | - Ye Zhu
- Robert D. and Patricia E. Kern Centre for the Science of Healthcare Delivery, Mayo Clinic, Rochester, MN USA
| | - Zhen Wang
- Robert D. and Patricia E. Kern Centre for the Science of Healthcare Delivery, Mayo Clinic, Rochester, MN USA
| | - Eduardo Antpack
- Division of Hospital Internal Medicine, Mayo Clinic Health System, Austin, MN USA
| | | | - Sandeep R. Pagali
- Division of Hospital Internal Medicine, Mayo Clinic, Rochester, MN USA
| | - Sanjeev Nanda
- Division of General Internal Medicine, Mayo Clinic, Rochester, MN USA
| | | | - Umesh M. Sharma
- Division of Hospital Internal Medicine, Mayo Clinic, Phoenix, AZ USA
| | - Sumit Bhagra
- Department of Endocrine and Metabolism, Mayo Clinic Health System, Austin, MN USA
| | - Sagar Dugani
- Division of Hospital Internal Medicine, Mayo Clinic, Rochester, MN USA
| | - Paul Y. Takahashi
- Division of Community Internal Medicine, Mayo Clinic, Rochester, MN USA
| | - Mohammad H. Murad
- Robert D. and Patricia E. Kern Centre for the Science of Healthcare Delivery, Mayo Clinic, Rochester, MN USA
- Division of Preventive Medicine, Mayo Clinic, Rochester, MN USA
| | - Mohammed Yousufuddin
- Division of Surgery, Mayo Clinic, Rochester, MN USA
- Hospital Internal Medicine, Mayo Clinic Health System, Mayo Clinic, 1000 1st Drive NW, Austin, MN USA
| |
Collapse
|
240
|
Olds PK, Musinguzi N, Geisler BP, Haberer JE. Evaluating disparities by social determinants in hospital admission decisions for patients with COVID-19 quaternary hospital early in the pandemic. Medicine (Baltimore) 2023; 102:e33178. [PMID: 36897732 PMCID: PMC9997198 DOI: 10.1097/md.0000000000033178] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/15/2022] [Accepted: 02/13/2023] [Indexed: 03/11/2023] Open
Abstract
The COVID-19 pandemic has highlighted significant disparities in hospital outcomes when focusing on social determinants of health. Better understanding the drivers of these disparities is not only critical for COVID-19 care but also to ensure equitable treatment more generally. In this paper, we look at how hospital admission patterns, both to the medical ward and the intensive care unit (ICU), may have differed by race, ethnicity, and social determinants of health. We conducted a retrospective chart review of all patients who presented to the Emergency Department of a large quaternary hospital between March 8 and June 3, 2020. We built logistic regression models to analyze how race, ethnicity, area deprivation index, English as a primary language, homelessness, and illicit substance use impacted the likelihood of admission while controlling for disease severity and timing of admission in relation to the start of data collection. We had 1302 recorded Emergency Department visits of patients diagnosed with SARS-CoV-2. White, Hispanic, and African American patients made up 39.2%, 37.5%, and 10.4% of the population respectively. Primary language was recorded as English for 41.2% and non-English for 30% of patients. Among the social determinants of health assessed, we found that illicit drug use significantly increased the likelihood for admission to the medical ward (odds ratio 4.4, confidence interval 1.1-17.1, P = .04), and that having a language other than English as a primary language significantly increased the likelihood of ICU admission (odds ratio 2.6, confidence interval 1.2-5.7, P = .02). Illicit drug use was associated with an increased likelihood of medical ward admission, potentially due to clinician concerns for complicated withdrawal or blood-stream infections from intravenous drug use. The increased likelihood of ICU admission associated with a primary language other than English may have been driven by communication difficulties or differences in disease severity that our model did not detect. Further work is required to better understand drivers of disparities in hospital COVID-19 care.
Collapse
Affiliation(s)
- Peter K. Olds
- Massachusetts General Hospital, Harvard Medical School, Boston, MA
| | | | - Benjamin P. Geisler
- Massachusetts General Hospital, Harvard Medical School, Boston, MA
- Ludwig Maximilian University Munich, Munich, Germany
| | - Jessica E. Haberer
- Massachusetts General Hospital, Harvard Medical School, Boston, MA
- Mbarara University of Science and Technology, Mbarara, Uganda
| |
Collapse
|
241
|
Mizrahi Reuveni M, Kertes J, Shapiro Ben David S, Shahar A, Shamir-Stein N, Rosen K, Liran O, Bar-Yishay M, Adler L. Risk Stratification Model for Severe COVID-19 Disease: A Retrospective Cohort Study. Biomedicines 2023; 11:biomedicines11030767. [PMID: 36979745 PMCID: PMC10045652 DOI: 10.3390/biomedicines11030767] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2023] [Revised: 02/19/2023] [Accepted: 02/26/2023] [Indexed: 03/06/2023] Open
Abstract
Background: Risk stratification models have been developed to identify patients that are at a higher risk of COVID-19 infection and severe illness. Objectives To develop and implement a scoring tool to identify COVID-19 patients that are at risk for severe illness during the Omicron wave. Methods: This is a retrospective cohort study that was conducted in Israel’s second-largest healthcare maintenance organization. All patients with a new episode of COVID-19 between 26 November 2021 and 18 January 2022 were included. A model was developed to predict severe illness (COVID-19-related hospitalization or death) based on one-third of the study population (the train group). The model was then applied to the remaining two-thirds of the study population (the test group). Risk score sensitivity, specificity, and positive predictive value rates, and receiver operating characteristics (ROC) were calculated to describe the performance of the model. Results: A total of 409,693 patients were diagnosed with COVID-19 over the two-month study period, of which 0.4% had severe illness. Factors that were associated with severe disease were age (age > 75, OR-70.4, 95% confidence interval [CI] 42.8–115.9), immunosuppression (OR-4.8, 95% CI 3.4–6.7), and pregnancy (5 months or more, OR-82.9, 95% CI 53–129.6). Factors that were associated with a reduced risk for severe disease were vaccination status (patients vaccinated in the previous six months OR-0.6, 95% CI 0.4–0.8) and a prior episode of COVID-19 (OR-0.3, 95% CI 0.2–0.5). According to the model, patients who were in the 10th percentile of the risk severity score were considered at an increased risk for severe disease. The model accuracy was 88.7%. Conclusions: This model has allowed us to prioritize patients requiring closer follow-up by their physicians and outreach services, as well as identify those that are most likely to benefit from anti-viral treatment during the fifth wave of infection in Israel, dominated by the Omicron variant.
Collapse
Affiliation(s)
| | - Jennifer Kertes
- Health Division, Maccabi Healthcare Services, Tel Aviv 6812509, Israel
| | - Shirley Shapiro Ben David
- Health Division, Maccabi Healthcare Services, Tel Aviv 6812509, Israel
- Department of Family Medicine, Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv 6997801, Israel
| | - Arnon Shahar
- Health Division, Maccabi Healthcare Services, Tel Aviv 6812509, Israel
| | | | - Keren Rosen
- Health Division, Maccabi Healthcare Services, Tel Aviv 6812509, Israel
| | - Ori Liran
- Health Division, Maccabi Healthcare Services, Tel Aviv 6812509, Israel
| | - Mattan Bar-Yishay
- Health Division, Maccabi Healthcare Services, Tel Aviv 6812509, Israel
| | - Limor Adler
- Health Division, Maccabi Healthcare Services, Tel Aviv 6812509, Israel
- Department of Family Medicine, Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv 6997801, Israel
- Correspondence:
| |
Collapse
|
242
|
Shukla AK, Seth T, Muhuri PK. Artificial intelligence centric scientific research on COVID-19: an analysis based on scientometrics data. MULTIMEDIA TOOLS AND APPLICATIONS 2023; 82:1-33. [PMID: 37362722 PMCID: PMC9978294 DOI: 10.1007/s11042-023-14642-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/07/2022] [Revised: 07/01/2022] [Accepted: 02/03/2023] [Indexed: 06/28/2023]
Abstract
With the spread of the deadly coronavirus disease throughout the geographies of the globe, expertise from every field has been sought to fight the impact of the virus. The use of Artificial Intelligence (AI), especially, has been the center of attention due to its capability to produce trustworthy results in a reasonable time. As a result, AI centric based research on coronavirus (or COVID-19) has been receiving growing attention from different domains ranging from medicine, virology, and psychiatry etc. We present this comprehensive study that closely monitors the impact of the pandemic on global research activities related exclusively to AI. In this article, we produce highly informative insights pertaining to publications, such as the best articles, research areas, most productive and influential journals, authors, and institutions. Studies are made on top 50 most cited articles to identify the most influential AI subcategories. We also study the outcome of research from different geographic areas while identifying the research collaborations that have had an impact. This study also compares the outcome of research from the different countries around the globe and produces insights on the same.
Collapse
Affiliation(s)
- Amit K. Shukla
- Faculty of Information Technology, University of Jyväskylä, Box 35 (Agora), Jyväskylä, 40014 Finland
| | - Taniya Seth
- Department of Computer Science, South Asian University, Akbar Bhawan, Chanakyapuri, New Delhi 110021 India
| | - Pranab K. Muhuri
- Department of Computer Science, South Asian University, Akbar Bhawan, Chanakyapuri, New Delhi 110021 India
| |
Collapse
|
243
|
Chirico I, Pappadà A, Giebel C, Ottoboni G, Valente M, Gabbay M, Chattat R. The impact of COVID-19 restrictions and care home strategies on residents with dementia as experienced by family carers in Italy. Aging Ment Health 2023; 27:512-520. [PMID: 35333142 DOI: 10.1080/13607863.2022.2056137] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/01/2022]
Abstract
OBJECTIVES The COVID-19 pandemic and public health measures caused serious consequences for several population cohorts, including people with dementia in care homes and their families. The aim of this study was to explore the impact of COVID-19 on care home residents with dementia as experienced by family carers in Italy. Specifically, strategies implemented to overcome the pandemic's constraints, their influence upon care, and consequences for everyday life of residents with dementia and carers were investigated. METHODS Semi-structured interviews explored participants' experiences of the pandemic, its restrictions and the services' status during lockdown. Transcripts were analysed via thematic analysis. RESULTS 26 family carers were interviewed. Three themes emerged: (1) COVID-19 restrictions negatively affected both residents with dementia and family carers, (2) Changing policies in care homes during COVID-19, and (3) Technology use in care homes during COVID-19. COVID-19 restrictions severely affected care home residents with dementia, disrupted their daily living, and accelerated their cognitive decline. Consequently carers' emotional burdens increased. Care home response strategies (safe visiting and digital solutions) were critical, though they were not enough to compensate for the lack of close in-person contacts. CONCLUSIONS Mixed evidence emerged about the feasibility of care home strategies and their associated benefits. To meet arising needs and possible future pandemic waves, there is a need for updated health strategies. These should prioritise a continuity of therapeutic activities and minimize negative effects on residents' quality of life, whilst incorporating feasible and accessible digital solutions to provide remote communication and psychological support for family carers.
Collapse
Affiliation(s)
| | | | - Clarissa Giebel
- Department of Primary Care & Mental Health, University of Liverpool, UK.,NIHR ARC NWC, Liverpool, UK
| | | | - Marco Valente
- Department of Psychology, University of Bologna, Italy
| | - Mark Gabbay
- Department of Primary Care & Mental Health, University of Liverpool, UK.,NIHR ARC NWC, Liverpool, UK
| | - Rabih Chattat
- Department of Psychology, University of Bologna, Italy
| |
Collapse
|
244
|
McDermott MBA, Nestor B, Szolovits P. Clinical Artificial Intelligence: Design Principles and Fallacies. Clin Lab Med 2023; 43:29-46. [PMID: 36764807 DOI: 10.1016/j.cll.2022.09.004] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Clinical artificial intelligence (AI)/machine learning (ML) is anticipated to offer new abilities in clinical decision support, diagnostic reasoning, precision medicine, clinical operational support, and clinical research, but careful concern is needed to ensure these technologies work effectively in the clinic. Here, we detail the clinical ML/AI design process, identifying several key questions and detailing several common forms of issues that arise with ML tools, as motivated by real-world examples, such that clinicians and researchers can better anticipate and correct for such issues in their own use of ML/AI techniques.
Collapse
Affiliation(s)
| | - Bret Nestor
- Department of Computer Science, University of Toronto, 40 St George St, Toronto, ON M5S 2E4, Canada
| | | |
Collapse
|
245
|
AlGhawi FS, AlMudarra SS, Assiri AM. Mortality patterns among COVID-19 patients in two Saudi hospitals: Demographics, etiology, and treatment. Influenza Other Respir Viruses 2023; 17:e13127. [PMID: 36970568 PMCID: PMC10030359 DOI: 10.1111/irv.13127] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Revised: 02/26/2023] [Accepted: 03/02/2023] [Indexed: 03/24/2023] Open
Abstract
Background Saudi Arabia (SA) reported its first case of COVID-19 on 2 March 2020. Mortality varied nationwide; by April 14, 2020, Medina had 16% of SA's total COVID-19 cases and 40% of all COVID-19 deaths. A team of epidemiologists investigated to identify factors impacting survival. Methods We reviewed medical records from two hospitals: Hospital A in Medina and Hospital B in Dammam. All patients with a registered COVID-related death between March and May 1, 2020, were included. We collected data on demographics, chronic health conditions, clinical presentation, and treatment. We analyzed data using SPSS. Results We identified 76 cases: 38 cases from each hospital. More fatalities were among non-Saudis at Hospital A (89%) versus Hospital B (82%, p < 0.001). Hypertension prevalence was higher among cases at Hospital B (42%) versus Hospital A (21%) (p < 0.05). We found statistically significant differences (p < 0.05) in symptoms at initial presentation among cases at Hospital B versus Hospital A, including body temperature (38°C vs. 37°C), heart rate (104 bpm vs. 89 bpm), and regular breathing rhythms (61% vs. 55%). Fewer cases (50%) at Hospital A received heparin versus Hospital B (97%, p-value < 0.001). Conclusion Patients who died typically presented with more severe illnesses and were more likely to have underlying health conditions. Migrant workers may be at increased risk due to poorer baseline health and reluctance to seek care. This highlights the importance of cross-cultural outreach to prevent deaths. Health education efforts should be multilingual and accommodate all literacy levels.
Collapse
Affiliation(s)
| | - Sami S. AlMudarra
- Field Epidemiology Training ProgramMinistry of HealthRiyadhSaudi Arabia
| | | |
Collapse
|
246
|
Chen H, Wang X, Lan X, Yu T, Li L, Tang S, Liu S, Jiang F, Wang L, Zhang J. A radiomics model development via the associations with genomics features in predicting axillary lymph node metastasis of breast cancer: a study based on a public database and single-centre verification. Clin Radiol 2023; 78:e279-e287. [PMID: 36623978 DOI: 10.1016/j.crad.2022.11.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2022] [Revised: 11/17/2022] [Accepted: 11/29/2022] [Indexed: 12/27/2022]
Abstract
AIM To evaluate the predictive performance of the radiomics model in predicting axillary lymph node (ALN) metastasis through the associations between radiomics features and genomic features in patients with breast cancer. MATERIALS AND METHODS Patients with breast cancer were enrolled retrospectively from a public database (111 patients as training group) and one hospital (15 patients as external validation group). The genomics features from transcriptome data and radiomics features from dynamic contrast-enhanced magnetic resonance imaging (MRI) were collected. Firstly, overlapping genes were identified using the Kyoto Encyclopedia of Genes and Genomes and differentially expressed gene analysis, while radiomics features were reduced using a data-driven method. Then, the associations between overlapping genes and retained radiomics features were assessed to obtain key pairs of radiomics-genomics features. Furthermore, the least absolute shrinkage and selection operator (LASSO) algorithm was used to detect the key-pairs features. Finally, radiomics and genomics models were constructed to predict ALN metastasis. RESULTS After using the hybrid data- and gene-driven selection method, key pairs of features were detected, which consisted of six radiomic features associated with four genomic features. The radiomics model exhibited comparable performance to the genomics model in predicting ALN metastasis (radiomic model: area under the curve [AUC] = 0.71, sensitivity = 77%, specificity = 56%; genomic model: AUC = 0.72, sensitivity = 85%, specificity = 74%). The four genomic features were enriched in six pathways and related to metabolism and human diseases. CONCLUSION The radiomics model established using the gene-driven hybrid selection method could predict ALN metastasis in breast cancer, which showed comparable performance to the genomics model.
Collapse
Affiliation(s)
- H Chen
- Department of Radiology, Chongqing University Cancer Hospital, Chongqing, PR China
| | - X Wang
- Department of Radiology, Chongqing University Cancer Hospital, Chongqing, PR China
| | - X Lan
- Department of Radiology, Chongqing University Cancer Hospital, Chongqing, PR China
| | - T Yu
- Department of Radiology, Chongqing University Cancer Hospital, Chongqing, PR China
| | - L Li
- Department of Radiology, Chongqing University Cancer Hospital, Chongqing, PR China
| | - S Tang
- Department of Radiology, Chongqing University Cancer Hospital, Chongqing, PR China
| | - S Liu
- Department of Radiology, Chongqing University Cancer Hospital, Chongqing, PR China
| | - F Jiang
- Department of Radiology, Chongqing University Cancer Hospital, Chongqing, PR China
| | - L Wang
- Department of Radiology, Chongqing University Cancer Hospital, Chongqing, PR China
| | - J Zhang
- Department of Radiology, Chongqing University Cancer Hospital, Chongqing, PR China.
| |
Collapse
|
247
|
Tracking Covid-19 cases and deaths in the United States: metrics of pandemic progression derived from a queueing framework. Health Care Manag Sci 2023; 26:79-92. [PMID: 36282367 PMCID: PMC9592548 DOI: 10.1007/s10729-022-09619-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2022] [Accepted: 09/26/2022] [Indexed: 11/04/2022]
Abstract
We analyze the progression of COVID-19 in the United States over a nearly one-year period beginning March 1, 2020 with a novel metric motivated by queueing models, tracking partial-average day-of-event and cumulative probability distributions for events, where events are points in time when new cases and new deaths are reported. The partial average represents the average day of all events preceding a point of time, and is an indicator as to whether the pandemic is accelerating or decelerating in the context of the entire history of the pandemic. The measure supplements traditional metrics, and also enables direct comparisons of case and death histories on a common scale. We also compare methods for estimating actual infections and deaths to assess the timing and dynamics of the pandemic by location. Three example states are graphically compared as functions of date, as well as Hong Kong as an example that experienced a pronounced recent wave of the pandemic. In addition, statistics are compared for all 50 states. Over the period studied, average case day and average death day varied by two to five months among the 50 states, depending on data source, with the earliest averages in New York and surrounding states, as well as Louisiana.
Collapse
|
248
|
Furuhata H, Araki K. Validation of a specialized evaluation system for COVID-19 in Japan: A retrospective, multicenter cohort study. J Infect Chemother 2023; 29:294-301. [PMID: 36529450 PMCID: PMC9753483 DOI: 10.1016/j.jiac.2022.12.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2022] [Revised: 11/22/2022] [Accepted: 12/07/2022] [Indexed: 12/23/2022]
Abstract
INTRODUCTION Evaluation of a severity grade (SG) is important to classify patients for efficient use of limited medical resources. This study validates two existing evaluation systems for the prevention of the coronavirus disease 2019 (COVID-19) in Japan: a criterion of SG and a list of 14 specialized underlying diseases (SUDs). METHODS A retrospective cohort was created using electronic medical records from 18 research institutes. The cohort includes 6,050 COVID-19 patients with two types of diagnosis information as follows: SG at hospitalization among mild, moderate I, moderate II, and severe and aggravation after hospitalization. RESULTS A crude mortality rate and an aggravation rate increased by the worsening of SG in the COVID-19 cohort. The transition of the aggravation rate was notable for COVID-19 patients with SUD. A conditional probability of the mortality given the aggravation in the COVID-19 cohort was 87.4% compared to mild or moderate patients (approximately 21%-45%) who have the possibility of the aggravation. An odds ratio of the mortality and aggravation information about the SUD list was higher than other variables. CONCLUSIONS We demonstrated the possibility of improving the criteria of SG by including the SUD list for more effective operation of the criteria of SG. Furthermore, we demonstrated the importance of the prevention of the aggravation based on the conditional probability, and the possibility of predicting the aggravation using the risk factors.
Collapse
Affiliation(s)
- Hiroki Furuhata
- Graduate School of Medicine and Veterinary Medicine, University of Miyazaki, 5200 Kibara Kiyotake-cho, Miyazaki, 8891692, Japan.
| | - Kenji Araki
- Department of Patient Advocacy Center, University of Miyazaki Hospital, 5200 Kibara Kiyotake-cho, Miyazaki, 8891692, Japan
| |
Collapse
|
249
|
Alhmoud B, Bonicci T, Patel R, Melley D, Hicks L, Banerjee A. Implementation of a digital early warning score (NEWS2) in a cardiac specialist and general hospital settings in the COVID-19 pandemic: a qualitative study. BMJ Open Qual 2023; 12:bmjoq-2022-001986. [PMID: 36914225 PMCID: PMC10015673 DOI: 10.1136/bmjoq-2022-001986] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2022] [Accepted: 03/02/2023] [Indexed: 03/16/2023] Open
Abstract
OBJECTIVES To evaluate implementation of digital National Early Warning Score 2 (NEWS2) in a cardiac care setting and a general hospital setting in the COVID-19 pandemic. DESIGN Thematic analysis of qualitative semistructured interviews using the non-adoption, abandonment, scale-up, spread, sustainability framework with purposefully sampled nurses and managers, as well as online surveys from March to December 2021. SETTINGS Specialist cardiac hospital (St Bartholomew's Hospital) and general teaching hospital (University College London Hospital, UCLH). PARTICIPANTS Eleven nurses and managers from cardiology, cardiac surgery, oncology and intensive care wards (St Bartholomew's) and medical, haematology and intensive care wards (UCLH) were interviewed and 67 were surveyed online. RESULTS Three main themes emerged: (1) implementing NEWS2 challenges and supports; (2) value of NEWS2 to alarm, escalate and during the pandemic; and (3) digitalisation: electronic health record (EHR) integration and automation. The value of NEWS2 was partly positive in escalation, yet there were concerns by nurses who undervalued NEWS2 particularly in cardiac care. Challenges, like clinicians' behaviours, lack of resources and training and the perception of NEWS2 value, limit the success of this implementation. Changes in guidelines in the pandemic have led to overlooking NEWS2. EHR integration and automated monitoring are improvement solutions that are not fully employed yet. CONCLUSION Whether in specialist or general medical settings, the health professionals implementing early warning score in healthcare face cultural and system-related challenges to adopting NEWS2 and digital solutions. The validity of NEWS2 in specialised settings and complex conditions is not yet apparent and requires comprehensive validation. EHR integration and automation are powerful tools to facilitate NEWS2 if its principles are reviewed and rectified, and resources and training are accessible. Further examination of implementation from the cultural and automation domains is needed.
Collapse
Affiliation(s)
- Baneen Alhmoud
- Institute of Health Informatics, University College London, London, UK.,University College London Hospitals NHS Foundation Trust, London, UK
| | - Timothy Bonicci
- Institute of Health Informatics, University College London, London, UK.,University College London Hospitals NHS Foundation Trust, London, UK
| | - Riyaz Patel
- University College London Hospitals NHS Foundation Trust, London, UK.,University College London, London, UK
| | | | | | - Amitava Banerjee
- Institute of Health Informatics, University College London, London, UK .,University College London Hospitals NHS Foundation Trust, London, UK
| |
Collapse
|
250
|
Maestro de la Calle G, García Reyne A, Lora-Tamayo J, Muiño Miguez A, Arnalich-Fernandez F, Beato Pérez JL, Vargas Núñez JA, Caudevilla Martínez MA, Alcalá Rivera N, Orviz Garcia E, Sánchez Moreno B, Freire Castro SJ, Rhyman N, Pesqueira Fontan PM, Piles L, López Caleya JF, Fraile Villarejo ME, Jiménez-García N, Boixeda R, González Noya A, Gracia Gutiérrez A, Martín Oterino JÁ, Gómez Huelgas R, Antón Santos JM, Lumbreras Bermejo C. Impact of days elapsed from the onset of symptoms to hospitalization in COVID-19 in-hospital mortality: time matters. REVISTA CLÍNICA ESPAÑOLA (ENGLISH EDITION) 2023; 223:281-297. [PMID: 36997085 PMCID: PMC10074179 DOI: 10.1016/j.rceng.2023.03.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/08/2022] [Accepted: 02/18/2023] [Indexed: 03/30/2023]
Abstract
BACKGROUND COVID-19 shows different clinical and pathophysiological stages over time. The effect of days elapsed from the onset of symptoms (DEOS) to hospitalization on COVID-19 prognostic factors remains uncertain. We analyzed the impact on mortality of DEOS to hospitalization and how other independent prognostic factors perform when taking this time elapsed into account. METHODS This retrospective, nationwide cohort study, included patients with confirmed COVID-19 from February 20th and May 6th, 2020. The data was collected in a standardized online data capture registry. Univariate and multivariate COX-regression were performed in the general cohort and the final multivariate model was subjected to a sensitivity analysis in an early presenting (EP; <5 DEOS) and late presenting (LP; ≥5 DEOS) group. RESULTS 7,915 COVID-19 patients were included in the analysis, 2,324 in the EP and 5,591 in the LP group. DEOS to hospitalization was an independent prognostic factor of in-hospital mortality in the multivariate Cox regression model along with other 9 variables. Each DEOS increment accounted for a 4,3% mortality risk reduction (HR 0.957; 95% CI 0.93 - 0.98). Regarding variations in other mortality predictors in the sensitivity analysis, the Charlson Comorbidity Index only remained significant in the EP group while D-dimer only remained significant in the LP group. CONCLUSION When caring for COVID-19 patients, DEOS to hospitalization should be considered as their need for early hospitalization confers a higher risk of mortality. Different prognostic factors vary over time and should be studied within a fixed timeframe of the disease.
Collapse
|