1
|
Rosati D, Palmieri M, Brunelli G, Morrione A, Iannelli F, Frullanti E, Giordano A. Differential gene expression analysis pipelines and bioinformatic tools for the identification of specific biomarkers: A review. Comput Struct Biotechnol J 2024; 23:1154-1168. [PMID: 38510977 PMCID: PMC10951429 DOI: 10.1016/j.csbj.2024.02.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2023] [Revised: 02/20/2024] [Accepted: 02/20/2024] [Indexed: 03/22/2024] Open
Abstract
In recent years, the role of bioinformatics and computational biology together with omics techniques and transcriptomics has gained tremendous importance in biomedicine and healthcare, particularly for the identification of biomarkers for precision medicine and drug discovery. Differential gene expression (DGE) analysis is one of the most used techniques for RNA-sequencing (RNA-seq) data analysis. This tool, which is typically used in various RNA-seq data processing applications, allows the identification of differentially expressed genes across two or more sample sets. Functional enrichment analyses can then be performed to annotate and contextualize the resulting gene lists. These studies provide valuable information about disease-causing biological processes and can help in identifying molecular targets for novel therapies. This review focuses on differential gene expression (DGE) analysis pipelines and bioinformatic techniques commonly used to identify specific biomarkers and discuss the advantages and disadvantages of these techniques.
Collapse
Affiliation(s)
- Diletta Rosati
- Department of Medical Biotechnologies, University of Siena, 53100 Siena, Italy
- Cancer Genomics & Systems Biology Lab, Dept. of Medical Biotechnologies, University of Siena, 53100 Siena, Italy
- Med Biotech Hub and Competence Center, Department of Medical Biotechnologies, University of Siena, Italy
| | - Maria Palmieri
- Cancer Genomics & Systems Biology Lab, Dept. of Medical Biotechnologies, University of Siena, 53100 Siena, Italy
- Med Biotech Hub and Competence Center, Department of Medical Biotechnologies, University of Siena, Italy
| | - Giulia Brunelli
- Med Biotech Hub and Competence Center, Department of Medical Biotechnologies, University of Siena, Italy
| | - Andrea Morrione
- Sbarro Institute for Cancer Research and Molecular Medicine, Center for Biotechnology, Department of Biology, College of Science and Technology, Temple University, Philadelphia, PA 19122, USA
| | - Francesco Iannelli
- Laboratory of Molecular Microbiology and Biotechnology, Department of Medical Biotechnologies, University of Siena, Siena, Italy
| | - Elisa Frullanti
- Cancer Genomics & Systems Biology Lab, Dept. of Medical Biotechnologies, University of Siena, 53100 Siena, Italy
- Med Biotech Hub and Competence Center, Department of Medical Biotechnologies, University of Siena, Italy
| | - Antonio Giordano
- Department of Medical Biotechnologies, University of Siena, 53100 Siena, Italy
- Sbarro Institute for Cancer Research and Molecular Medicine, Center for Biotechnology, Department of Biology, College of Science and Technology, Temple University, Philadelphia, PA 19122, USA
| |
Collapse
|
2
|
Casal A, Suárez-Antelo J, Gude F, Lado-Baleato Ó, Otero B, Toubes ME, Ferreiro L, Rodríguez-Núñez N, Valdés L. Use of mycophenolate mofetil for the treatment of fibrotic hypersensitivity pneumonitis. Am J Med Sci 2024:S0002-9629(24)01356-9. [PMID: 39009283 DOI: 10.1016/j.amjms.2024.07.021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2024] [Revised: 07/10/2024] [Accepted: 07/10/2024] [Indexed: 07/17/2024]
Abstract
INTRODUCTION The optimal treatment of fibrosing hypersensitivity pneumonitis (fHP) is not well understood. The aim of the study was to obtain information about the usefulness of mycophenolate mofetil (MMF) in its treatment. MATERIAL AND METHODS Quasi-experimental analysis of patients diagnosed with fHP and treated with MMF for one year, in a single centre. From the start of treatment, data collection was prospective. RESULTS 73 were included and 58 completed the study. FVC% and DLCO% decreased until starting MMF (year -1 to year 0). After completion of treatment (year 1), FVC% stabilised (p=0.336) and DLCO% improved significantly (p=0.004) compared to year 0. Dyspnoea, number of patients without corticosteroids and mean corticosteroid dose also improved significantly (p<0.001 in all cases). Being male and having a history of tuberculosis were predictors of poor drug response [AUC = 0.89 (95% CI: 0.80-0.98)]. 45 adverse effects were observed in 34 patients (46.6%). In 4 cases (5.5%), the adverse effect was severe and required discontinuation of treatment. CONCLUSIONS In patients with fHP, MMF improves lung function and dyspnoea and reduces both the number of patients requiring oral corticosteroids and their mean dose in those who completed 1 year of treatment. The model constructed predicts which patients will respond poorly to treatment, with good discriminative ability and only a small percentage of patients will not tolerate treatment. Further prospective, randomised clinical trials are needed to define the role of this treatment in fHP.
Collapse
Affiliation(s)
- Ana Casal
- Pulmonology Department. Clinical-University Hospital of Santiago de Compostela. Santiago de Compostela, Spain.
| | - Juan Suárez-Antelo
- Pulmonology Department. Clinical-University Hospital of Santiago de Compostela. Santiago de Compostela, Spain
| | - Francisco Gude
- Concepción Arenal Primary Care Center, Santiago de Compostela, Spain; Health Research Institute of Santiago de Compostela (IDIS), Santiago de Compostela, Spain; Department of Medicine, Faculty of Medicine, University of Santiago de Compostela, Spain; ISCIII Support Platforms for Clinical Research, Spain
| | - Óscar Lado-Baleato
- Health Research Institute of Santiago de Compostela (IDIS), Santiago de Compostela, Spain; ISCIII Support Platforms for Clinical Research, Spain
| | | | - María E Toubes
- Pulmonology Department. Clinical-University Hospital of Santiago de Compostela. Santiago de Compostela, Spain
| | - Lucía Ferreiro
- Pulmonology Department. Clinical-University Hospital of Santiago de Compostela. Santiago de Compostela, Spain
| | - Nuria Rodríguez-Núñez
- Pulmonology Department. Clinical-University Hospital of Santiago de Compostela. Santiago de Compostela, Spain
| | - Luis Valdés
- Pulmonology Department. Clinical-University Hospital of Santiago de Compostela. Santiago de Compostela, Spain; Health Research Institute of Santiago de Compostela (IDIS), Santiago de Compostela, Spain; Department of Medicine, Faculty of Medicine, University of Santiago de Compostela, Spain
| |
Collapse
|
3
|
Prescott HC, Heath M, Munroe ES, Blamoun J, Bozyk P, Hechtman RK, Horowitz JK, Jayaprakash N, Kocher KE, Younas M, Taylor SP, Posa PJ, McLaughlin E, Flanders SA. Development and Validation of the Hospital Medicine Safety Sepsis Initiative Mortality Model. Chest 2024:S0012-3692(24)04571-9. [PMID: 38964673 DOI: 10.1016/j.chest.2024.06.3769] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2024] [Revised: 06/12/2024] [Accepted: 06/15/2024] [Indexed: 07/06/2024] Open
Abstract
BACKGROUND When comparing outcomes after sepsis, it is essential to account for patient case mix to make fair comparisons. We developed a model to assess risk-adjusted 30-day mortality in the Michigan Hospital Medicine Safety sepsis initiative (HMS-Sepsis). RESEARCH QUESTION Can HMS-Sepsis registry data adequately predict risk of 30-day mortality? Do performance assessments using adjusted vs unadjusted data differ? STUDY DESIGN AND METHODS Retrospective cohort of community-onset sepsis hospitalizations in the HMS-Sepsis registry (April 2022-September 2023), with split-derivation (70%) and validation (30%) cohorts. We fit a risk-adjustment model (HMS-Sepsis mortality model) incorporating acute physiologic, demographic, and baseline health data and assessed model performance using concordance (C) statistics, Brier's scores, and comparisons of predicted vs observed mortality by deciles of risk. We compared hospital performance (first quintile, middle quintiles, fifth quintile) using observed vs adjusted mortality to understand the extent to which risk adjustment impacted hospital performance assessment. RESULTS Among 17,514 hospitalizations from 66 hospitals during the study period, 12,260 hospitalizations (70%) were used for model derivation and 5,254 hospitalizations (30%) were used for model validation. Thirty-day mortality for the total cohort was 19.4%. The final model included 13 physiologic variables, two physiologic interactions, and 16 demographic and chronic health variables. The most significant variables were age, metastatic solid tumor, temperature, altered mental status, and platelet count. The model C statistic was 0.82 for the derivation cohort, 0.81 for the validation cohort, and ≥ 0.78 for all subgroups assessed. Overall calibration error was 0.0%, and mean calibration error across deciles of risk was 1.5%. Standardized mortality ratios yielded different assessments than observed mortality for 33.9% of hospitals. INTERPRETATION The HMS-Sepsis mortality model showed strong discrimination and adequate calibration and reclassified one-third of hospitals to a different performance category from unadjusted mortality. Based on its strong performance, the HMS-Sepsis mortality model can aid in fair hospital benchmarking, assessment of temporal changes, and observational causal inference analysis.
Collapse
Affiliation(s)
- Hallie C Prescott
- Department of Internal Medicine, University of Michigan, Ann Arbor, MI; VA Center for Clinical Management Research, Ann Arbor, MI.
| | - Megan Heath
- Department of Internal Medicine, University of Michigan, Ann Arbor, MI
| | | | | | | | - Rachel K Hechtman
- Department of Internal Medicine, University of Michigan, Ann Arbor, MI
| | | | | | - Keith E Kocher
- Department of Emergency Medicine, University of Michigan, Ann Arbor, MI; VA Center for Clinical Management Research, Ann Arbor, MI
| | | | | | - Patricia J Posa
- Department of Internal Medicine, University of Michigan, Ann Arbor, MI
| | | | - Scott A Flanders
- Department of Internal Medicine, University of Michigan, Ann Arbor, MI
| |
Collapse
|
4
|
Gutman R, Karavani E, Shimoni Y. Improving Inverse Probability Weighting by Post-calibrating Its Propensity Scores. Epidemiology 2024; 35:473-480. [PMID: 38619218 PMCID: PMC11191550 DOI: 10.1097/ede.0000000000001733] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Accepted: 03/18/2024] [Indexed: 04/16/2024]
Abstract
Theoretical guarantees for causal inference using propensity scores are partially based on the scores behaving like conditional probabilities. However, prediction scores between zero and one do not necessarily behave like probabilities, especially when output by flexible statistical estimators. We perform a simulation study to assess the error in estimating the average treatment effect before and after applying a simple and well-established postprocessing method to calibrate the propensity scores. We observe that postcalibration reduces the error in effect estimation and that larger improvements in calibration result in larger improvements in effect estimation. Specifically, we find that expressive tree-based estimators, which are often less calibrated than logistic regression-based models initially, tend to show larger improvements relative to logistic regression-based models. Given the improvement in effect estimation and that postcalibration is computationally cheap, we recommend its adoption when modeling propensity scores with expressive models.
Collapse
Affiliation(s)
- Rom Gutman
- From the IBM Research, University of Haifa Campus
- Technion - Israel Institute of Technology, Haifa, Israel
| | | | | |
Collapse
|
5
|
Zou W, Yao X, Chen Y, Li X, Huang J, Zhang Y, Yu L, Xie B. An elastic net regression model for predicting the risk of ICU admission and death for hospitalized patients with COVID-19. Sci Rep 2024; 14:14404. [PMID: 38909101 PMCID: PMC11193779 DOI: 10.1038/s41598-024-64776-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2024] [Accepted: 06/12/2024] [Indexed: 06/24/2024] Open
Abstract
This study aimed to develop and validate prediction models to estimate the risk of death and intensive care unit admission in COVID-19 inpatients. All RT-PCR-confirmed adult COVID-19 inpatients admitted to Fujian Provincial Hospital from October 2022 to April 2023 were considered. Elastic Net Regression was used to derive the risk prediction models. Potential risk factors were considered, which included demographic characteristics, clinical symptoms, comorbidities, laboratory results, treatment process, prognosis. A total of 1906 inpatients were included finally by inclusion/exclusion criteria and were divided into derivation and test cohorts in a ratio of 8:2, where 1526 (80%) samples were used to develop prediction models under a repeated cross-validation framework and the remaining 380 (20%) samples were used for performance evaluation. Overall performance, discrimination and calibration were evaluated in the validation set and test cohort and quantified by accuracy, scaled Brier score (SbrS), the area under the ROC curve (AUROC), and Spiegelhalter-Z statistics. The models performed well, with high levels of discrimination (AUROCICU [95%CI]: 0.858 [0.803,0.899]; AUROCdeath [95%CI]: 0.906 [0.850,0.948]); and good calibrations (Spiegelhalter-ZICU: - 0.821 (p-value: 0.412); Spiegelhalter-Zdeath: 0.173) in the test set. We developed and validated prediction models to help clinicians identify high risk patients for death and ICU admission after COVID-19 infection.
Collapse
Affiliation(s)
- Wei Zou
- Shengli Clinical Medical College of Fujian Medical University, Fuzhou, 350013, China
- Department of Pulmonary and Critical Care Medicine, Fujian Provincial Hospital, Fuzhou, 350004, China
| | - Xiujuan Yao
- Shengli Clinical Medical College of Fujian Medical University, Fuzhou, 350013, China
- Department of Pulmonary and Critical Care Medicine, Fujian Provincial Hospital, Fuzhou, 350004, China
| | - Yizhen Chen
- Shengli Clinical Medical College of Fujian Medical University, Fuzhou, 350013, China
| | - Xiaoqin Li
- Shengli Clinical Medical College of Fujian Medical University, Fuzhou, 350013, China
- Department of Pulmonary and Critical Care Medicine, Fujian Provincial Hospital, Fuzhou, 350004, China
| | - Jiandong Huang
- Shengli Clinical Medical College of Fujian Medical University, Fuzhou, 350013, China
| | - Yong Zhang
- Chongqing Nanpeng Artificial Intelligence Technology Research Institute Co., Ltd., Chongqing, 401123, China
| | - Lin Yu
- Chongqing Nanpeng Artificial Intelligence Technology Research Institute Co., Ltd., Chongqing, 401123, China
| | - Baosong Xie
- Department of Pulmonary and Critical Care Medicine, Fujian Provincial Hospital, Fuzhou, 350004, China.
| |
Collapse
|
6
|
Beil M, Moreno R, Fronczek J, Kogan Y, Moreno RPJ, Flaatten H, Guidet B, de Lange D, Leaver S, Nachshon A, van Heerden PV, Joskowicz L, Sviri S, Jung C, Szczeklik W. Prognosticating the outcome of intensive care in older patients-a narrative review. Ann Intensive Care 2024; 14:97. [PMID: 38907141 PMCID: PMC11192712 DOI: 10.1186/s13613-024-01330-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2024] [Accepted: 06/10/2024] [Indexed: 06/23/2024] Open
Abstract
Prognosis determines major decisions regarding treatment for critically ill patients. Statistical models have been developed to predict the probability of survival and other outcomes of intensive care. Although they were trained on the characteristics of large patient cohorts, they often do not represent very old patients (age ≥ 80 years) appropriately. Moreover, the heterogeneity within this particular group impairs the utility of statistical predictions for informing decision-making in very old individuals. In addition to these methodological problems, the diversity of cultural attitudes, available resources as well as variations of legal and professional norms limit the generalisability of prediction models, especially in patients with complex multi-morbidity and pre-existing functional impairments. Thus, current approaches to prognosticating outcomes in very old patients are imperfect and can generate substantial uncertainty about optimal trajectories of critical care in the individual. This article presents the state of the art and new approaches to predicting outcomes of intensive care for these patients. Special emphasis has been given to the integration of predictions into the decision-making for individual patients. This requires quantification of prognostic uncertainty and a careful alignment of decisions with the preferences of patients, who might prioritise functional outcomes over survival. Since the performance of outcome predictions for the individual patient may improve over time, time-limited trials in intensive care may be an appropriate way to increase the confidence in decisions about life-sustaining treatment.
Collapse
Affiliation(s)
- Michael Beil
- Department of Medical Intensive Care, Hadassah Medical Center and Faculty of Medicine, Hebrew University of Jerusalem, Jerusalem, Israel
| | - Rui Moreno
- Unidade Local de Saúde São José, Hospital de São José, Lisbon, Portugal
- Centro Clínico Académico de Lisboa, Lisbon, Portugal
- Faculdade de Ciências da Saúde, Universidade da Beira Interior, Covilhã, Portugal
| | - Jakub Fronczek
- Center for Intensive Care and Perioperative Medicine, Jagiellonian University Medical College, Krakow, Poland
| | - Yuri Kogan
- Institute for Medical Biomathematics, Bene Ataroth, Israel
| | | | - Hans Flaatten
- Department of Research and Development, Haukeland University Hospital, Bergen, Norway
| | - Bertrand Guidet
- INSERM, Institut Pierre Louis d'Epidémiologie Et de Santé Publique, AP-HP, Hôpital Saint Antoine, Sorbonne Université, Service MIR, Paris, France
| | - Dylan de Lange
- Department of Intensive Care Medicine, University Medical Center, University Utrecht, Utrecht, The Netherlands
| | - Susannah Leaver
- General Intensive Care, St George's University Hospitals NHS Foundation Trust, London, UK
| | - Akiva Nachshon
- General Intensive Care Unit, Department of Anaesthesiology, Critical Care and Pain Medicine, Faculty of Medicine, Hebrew University and, Hadassah University Medical Center, Jerusalem, Israel
| | - Peter Vernon van Heerden
- General Intensive Care Unit, Department of Anaesthesiology, Critical Care and Pain Medicine, Faculty of Medicine, Hebrew University and, Hadassah University Medical Center, Jerusalem, Israel
| | - Leo Joskowicz
- School of Computer Science and Engineering and Center for Computational Medicine, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Sigal Sviri
- Department of Medical Intensive Care, Hadassah Medical Center and Faculty of Medicine, Hebrew University of Jerusalem, Jerusalem, Israel
| | - Christian Jung
- Department of Cardiology, Pulmonology and Vascular Medicine, Faculty of Medicine, Heinrich-Heine-University, University Duesseldorf, Moorenstraße 5, 40225, Düsseldorf, Germany.
| | - Wojciech Szczeklik
- Center for Intensive Care and Perioperative Medicine, Jagiellonian University Medical College, Krakow, Poland
| |
Collapse
|
7
|
Luu J, Borisenko E, Przekop V, Patil A, Forrester JD, Choi J. Practical guide to building machine learning-based clinical prediction models using imbalanced datasets. Trauma Surg Acute Care Open 2024; 9:e001222. [PMID: 38881829 PMCID: PMC11177772 DOI: 10.1136/tsaco-2023-001222] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Accepted: 04/09/2024] [Indexed: 06/18/2024] Open
Abstract
Clinical prediction models often aim to predict rare, high-risk events, but building such models requires robust understanding of imbalance datasets and their unique study design considerations. This practical guide highlights foundational prediction model principles for surgeon-data scientists and readers who encounter clinical prediction models, from feature engineering and algorithm selection strategies to model evaluation and design techniques specific to imbalanced datasets. We walk through a clinical example using readable code to highlight important considerations and common pitfalls in developing machine learning-based prediction models. We hope this practical guide facilitates developing and critically appraising robust clinical prediction models for the surgical community.
Collapse
Affiliation(s)
- Jacklyn Luu
- Stanford University, Stanford, California, USA
| | | | | | | | | | - Jeff Choi
- Department of Surgery, Stanford University, Stanford, California, USA
| |
Collapse
|
8
|
Zhuang Q, Zhang AY, Cong RSTY, Yang GM, Neo PSH, Tan DS, Chua ML, Tan IB, Wong FY, Eng Hock Ong M, Shao Wei Lam S, Liu N. Towards proactive palliative care in oncology: developing an explainable EHR-based machine learning model for mortality risk prediction. BMC Palliat Care 2024; 23:124. [PMID: 38769564 PMCID: PMC11103848 DOI: 10.1186/s12904-024-01457-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Accepted: 05/15/2024] [Indexed: 05/22/2024] Open
Abstract
BACKGROUND Ex-ante identification of the last year in life facilitates a proactive palliative approach. Machine learning models trained on electronic health records (EHR) demonstrate promising performance in cancer prognostication. However, gaps in literature include incomplete reporting of model performance, inadequate alignment of model formulation with implementation use-case, and insufficient explainability hindering trust and adoption in clinical settings. Hence, we aim to develop an explainable machine learning EHR-based model that prompts palliative care processes by predicting for 365-day mortality risk among patients with advanced cancer within an outpatient setting. METHODS Our cohort consisted of 5,926 adults diagnosed with Stage 3 or 4 solid organ cancer between July 1, 2017, and June 30, 2020 and receiving ambulatory cancer care within a tertiary center. The classification problem was modelled using Extreme Gradient Boosting (XGBoost) and aligned to our envisioned use-case: "Given a prediction point that corresponds to an outpatient cancer encounter, predict for mortality within 365-days from prediction point, using EHR data up to 365-days prior." The model was trained with 75% of the dataset (n = 39,416 outpatient encounters) and validated on a 25% hold-out dataset (n = 13,122 outpatient encounters). To explain model outputs, we used Shapley Additive Explanations (SHAP) values. Clinical characteristics, laboratory tests and treatment data were used to train the model. Performance was evaluated using area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC), while model calibration was assessed using the Brier score. RESULTS In total, 17,149 of the 52,538 prediction points (32.6%) had a mortality event within the 365-day prediction window. The model demonstrated an AUROC of 0.861 (95% CI 0.856-0.867) and AUPRC of 0.771. The Brier score was 0.147, indicating slight overestimations of mortality risk. Explanatory diagrams utilizing SHAP values allowed visualization of feature impacts on predictions at both the global and individual levels. CONCLUSION Our machine learning model demonstrated good discrimination and precision-recall in predicting 365-day mortality risk among individuals with advanced cancer. It has the potential to provide personalized mortality predictions and facilitate earlier integration of palliative care.
Collapse
Affiliation(s)
- Qingyuan Zhuang
- Division of Supportive and Palliative Care, National Cancer Centre Singapore, 30 Hospital Blvd, Singapore, 168583, Singapore.
- Data Computational Science Core, National Cancer Centre Singapore, Singapore, Singapore.
| | - Alwin Yaoxian Zhang
- Division of Supportive and Palliative Care, National Cancer Centre Singapore, 30 Hospital Blvd, Singapore, 168583, Singapore
| | - Ryan Shea Tan Ying Cong
- Division of Medical Oncology, National Cancer Centre Singapore, Singapore, Singapore
- Data Computational Science Core, National Cancer Centre Singapore, Singapore, Singapore
| | - Grace Meijuan Yang
- Division of Supportive and Palliative Care, National Cancer Centre Singapore, 30 Hospital Blvd, Singapore, 168583, Singapore
- Lien Centre of Palliative Care, Duke-NUS Medical School, Singapore, Singapore
| | - Patricia Soek Hui Neo
- Division of Supportive and Palliative Care, National Cancer Centre Singapore, 30 Hospital Blvd, Singapore, 168583, Singapore
| | - Daniel Sw Tan
- Division of Medical Oncology, National Cancer Centre Singapore, Singapore, Singapore
- Division of Clinical Trials and Epidemiological Sciences, National Cancer Centre Singapore, Singapore, Singapore
| | - Melvin Lk Chua
- Data Computational Science Core, National Cancer Centre Singapore, Singapore, Singapore
- Division of Radiation Oncology, National Cancer Centre Singapore, Singapore, Singapore
| | - Iain Beehuat Tan
- Division of Medical Oncology, National Cancer Centre Singapore, Singapore, Singapore
- Data Computational Science Core, National Cancer Centre Singapore, Singapore, Singapore
- Saw Swee Hock School of Public Health, National University of Singapore, Singapore, Singapore
| | - Fuh Yong Wong
- Division of Radiation Oncology, National Cancer Centre Singapore, Singapore, Singapore
- Department of Cancer Informatics, National Cancer Centre Singapore, Singapore, Singapore
| | - Marcus Eng Hock Ong
- Health Services Research Centre, SingHealth, Singapore
- Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
| | - Sean Shao Wei Lam
- Health Services Research Centre, SingHealth, Singapore
- Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
| | - Nan Liu
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
- Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
| |
Collapse
|
9
|
MacCarthy G, Pazoki R. Using Machine Learning to Evaluate the Value of Genetic Liabilities in the Classification of Hypertension within the UK Biobank. J Clin Med 2024; 13:2955. [PMID: 38792496 PMCID: PMC11122671 DOI: 10.3390/jcm13102955] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2024] [Revised: 05/01/2024] [Accepted: 05/07/2024] [Indexed: 05/26/2024] Open
Abstract
Background and Objective: Hypertension increases the risk of cardiovascular diseases (CVD) such as stroke, heart attack, heart failure, and kidney disease, contributing to global disease burden and premature mortality. Previous studies have utilized statistical and machine learning techniques to develop hypertension prediction models. Only a few have included genetic liabilities and evaluated their predictive values. This study aimed to develop an effective hypertension classification model and investigate the potential influence of genetic liability for multiple risk factors linked to CVD on hypertension risk using the random forest and the neural network. Materials and Methods: The study involved 244,718 European participants, who were divided into training and testing sets. Genetic liabilities were constructed using genetic variants associated with CVD risk factors obtained from genome-wide association studies (GWAS). Various combinations of machine learning models before and after feature selection were tested to develop the best classification model. The models were evaluated using area under the curve (AUC), calibration, and net reclassification improvement in the testing set. Results: The models without genetic liabilities achieved AUCs of 0.70 and 0.72 using the random forest and the neural network methods, respectively. Adding genetic liabilities improved the AUC for the random forest but not for the neural network. The best classification model was achieved when feature selection and classification were performed using random forest (AUC = 0.71, Spiegelhalter z score = 0.10, p-value = 0.92, calibration slope = 0.99). This model included genetic liabilities for total cholesterol and low-density lipoprotein (LDL). Conclusions: The study highlighted that incorporating genetic liabilities for lipids in a machine learning model may provide incremental value for hypertension classification beyond baseline characteristics.
Collapse
Affiliation(s)
- Gideon MacCarthy
- Cardiovascular and Metabolic Research Group, Division of Biomedical Sciences, Department of Life Sciences, College of Health, Medicine and Life Sciences, Brunel University London, London UB8 3PH, UK
| | - Raha Pazoki
- Cardiovascular and Metabolic Research Group, Division of Biomedical Sciences, Department of Life Sciences, College of Health, Medicine and Life Sciences, Brunel University London, London UB8 3PH, UK
- MRC Centre for Environment and Health, Department of Epidemiology and Biostatistics, School of Public Health, St Mary’s Campus, Norfolk Place, Imperial College London, London W2 1PG, UK
| |
Collapse
|
10
|
Staerk C, Klinkhammer H, Wistuba T, Maj C, Mayr A. Generalizability of polygenic prediction models: how is the R 2 defined on test data? BMC Med Genomics 2024; 17:132. [PMID: 38755654 PMCID: PMC11100126 DOI: 10.1186/s12920-024-01905-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2023] [Accepted: 05/08/2024] [Indexed: 05/18/2024] Open
Abstract
BACKGROUND Polygenic risk scores (PRS) quantify an individual's genetic predisposition for different traits and are expected to play an increasingly important role in personalized medicine. A crucial challenge in clinical practice is the generalizability and transferability of PRS models to populations with different ancestries. When assessing the generalizability of PRS models for continuous traits, the R 2 is a commonly used measure to evaluate prediction accuracy. While the R 2 is a well-defined goodness-of-fit measure for statistical linear models, there exist different definitions for its application on test data, which complicates interpretation and comparison of results. METHODS Based on large-scale genotype data from the UK Biobank, we compare three definitions of the R 2 on test data for evaluating the generalizability of PRS models to different populations. Polygenic models for several phenotypes, including height, BMI and lipoprotein A, are derived based on training data with European ancestry using state-of-the-art regression methods and are evaluated on various test populations with different ancestries. RESULTS Our analysis shows that the choice of the R 2 definition can lead to considerably different results on test data, making the comparison of R 2 values from the literature problematic. While the definition as the squared correlation between predicted and observed phenotypes solely addresses the discriminative performance and always yields values between 0 and 1, definitions of the R 2 based on the mean squared prediction error (MSPE) with reference to intercept-only models assess both discrimination and calibration. These MSPE-based definitions can yield negative values indicating miscalibrated predictions for out-of-target populations. We argue that the choice of the most appropriate definition depends on the aim of PRS analysis - whether it primarily serves for risk stratification or also for individual phenotype prediction. Moreover, both correlation-based and MSPE-based definitions of R 2 can provide valuable complementary information. CONCLUSIONS Awareness of the different definitions of the R 2 on test data is necessary to facilitate the reporting and interpretation of results on PRS generalizability. It is recommended to explicitly state which definition was used when reporting R 2 values on test data. Further research is warranted to develop and evaluate well-calibrated polygenic models for diverse populations.
Collapse
Affiliation(s)
- Christian Staerk
- Department of Medical Biometry, Informatics and Epidemiology, Medical Faculty, University of Bonn, Bonn, Germany.
- Institute of Statistics, RWTH Aachen University, Aachen, Germany.
| | - Hannah Klinkhammer
- Department of Medical Biometry, Informatics and Epidemiology, Medical Faculty, University of Bonn, Bonn, Germany
- Institute for Genomic Statistics and Bioinformatics, Medical Faculty, University of Bonn, Bonn, Germany
| | - Tobias Wistuba
- Department of Medical Biometry, Informatics and Epidemiology, Medical Faculty, University of Bonn, Bonn, Germany
| | - Carlo Maj
- Center for Human Genetics, University of Marburg, Marburg, Germany
| | - Andreas Mayr
- Department of Medical Biometry, Informatics and Epidemiology, Medical Faculty, University of Bonn, Bonn, Germany
| |
Collapse
|
11
|
Vranic I, Stankovic I, Ignjatovic A, Kafedzic S, Radovanovic-Radosavljevic M, Neskovic AN, Vidakovic R. Validation of the European Society of Cardiology pretest probability models for obstructive coronary artery disease in high-risk population. Hellenic J Cardiol 2024:S1109-9666(24)00107-6. [PMID: 38729349 DOI: 10.1016/j.hjc.2024.05.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2024] [Revised: 04/21/2024] [Accepted: 05/03/2024] [Indexed: 05/12/2024] Open
Abstract
OBJECTIVE The pre-test probability (PTP) model for obstructive coronary artery disease (CAD) was updated in 2019 by the European Society of Cardiology (ESC). To our knowledge, this model was never externally validated in a population with a high incidence of CAD. The aim of this study is to validate the new PTP ESC model in our population, which has a high CAD incidence, and to compare it with the previous PTP ESC model from 2013. METHODS We retrospectively analysed 1294 symptomatic patients with suspected CAD referred to our centre between 2015 and 2019. In all patients, the PTP score was calculated based on age, gender, and symptoms according to the ESC model from 2013 (2013-ESC-PTP) and 2019 (2019-ESC-PTP). All patients underwent invasive coronary angiography (ICA). RESULTS Of the 1294 patients, obstructive CAD was diagnosed in 533 patients (41.2%). The 2019-ESC-PTP model categorised significantly more patients into the low probability group (PTP < 15%) than the 2013-ESC-PTP model (39.8% vs. 5.6%, p < 0.001). Obstructive CAD prevalence was underestimated using 2019-ESC-PTP at all PTP levels (calibration intercept 1.15, calibration slope 0.96). The 2013-ESC-PTP overestimated obstructive CAD prevalence (calibration intercept -0.24, calibration slope 0.73). The discrimination measured with an area under the curve was similar for both models, indicating moderate accuracy of the models. CONCLUSION In high-risk Serbian population, both the 2013 and 2019 ESC-PTP models had moderate accuracy in diagnosing CAD, with the 2019-ESC-PTP underestimating the prevalence of CAD and the 2013-ESC-PTP overestimating it. Further studies are warranted to establish PTP models for high-risk countries.
Collapse
Affiliation(s)
- Ivona Vranic
- Clinical Hospital Centre Zemun, Department of Cardiology, Vukova 9, Belgrade 11 000, Serbia.
| | - Ivan Stankovic
- Clinical Hospital Centre Zemun, Department of Cardiology, Vukova 9, Belgrade 11 000, Serbia; Faculty of Medicine, University of Belgrade, Dr Subotica 8, Belgrade 11 000, Serbia
| | - Aleksandra Ignjatovic
- Medical Faculty, University of Nis, Department of Medical Statistics, Bul. Dr Zorana Djindjica 81, Nis 18000, Serbia
| | - Srdjan Kafedzic
- Clinical Hospital Centre Zemun, Department of Cardiology, Vukova 9, Belgrade 11 000, Serbia; Faculty of Medicine, University of Belgrade, Dr Subotica 8, Belgrade 11 000, Serbia
| | - Mina Radovanovic-Radosavljevic
- Faculty of Medicine, University of Belgrade, Dr Subotica 8, Belgrade 11 000, Serbia; University Clinical Centre Serbia, Emergency Department, Coronary Care Unit, Pasterova 2, Belgrade 11 000, Serbia
| | - Aleksandar N Neskovic
- Clinical Hospital Centre Zemun, Department of Cardiology, Vukova 9, Belgrade 11 000, Serbia; Faculty of Medicine, University of Belgrade, Dr Subotica 8, Belgrade 11 000, Serbia
| | - Radosav Vidakovic
- Clinical Hospital Centre Zemun, Department of Cardiology, Vukova 9, Belgrade 11 000, Serbia; Faculty of Medicine, University of Belgrade, Dr Subotica 8, Belgrade 11 000, Serbia
| |
Collapse
|
12
|
Zhou D, Mao Q, Sun Y, Cheng H, Zhao J, Liu Q, Deng M, Xu S, Zhao X. Association of Blood Copper With the Subclinical Carotid Atherosclerosis: An Observational Study. J Am Heart Assoc 2024; 13:e033474. [PMID: 38700020 PMCID: PMC11179917 DOI: 10.1161/jaha.123.033474] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Accepted: 04/01/2024] [Indexed: 05/05/2024]
Abstract
BACKGROUND Copper exposure is reported to be associated with increased risk of stroke. However, the association of copper exposure with subclinical carotid atherosclerosis remains unclear. METHODS AND RESULTS This observational study included consecutive participants from Xinqiao Hospital between May 2020 and August 2021. Blood metals were measured using inductively coupled plasma mass spectrometry and carotid atherosclerosis was assessed using ultrasound. Modified Poisson regression was performed to evaluate the associations of copper and other metals with subclinical carotid plaque presence. Blood metals were analyzed as categorical according to the quartiles. Multivariable models were adjusted for age, sex, body mass index, education, smoking, drinking, hypertension, diabetes, dyslipidemia, estimated glomerular filtration rate, and coronary artery disease history. Bayesian Kernel Machine Regression was conducted to evaluate the overall association of metal mixture with subclinical carotid plaque presence. One thousand five hundred eighty-five participants were finally enrolled in our study, and carotid plaque was found in 1091 subjects. After adjusting for potential confounders, metal-progressively-adjusted models showed that blood copper was positively associated with subclinical carotid plaque (relative risk according to comparing quartile 4 to quartile 1 was 1.124 [1.021-1.238], relative risk according to per interquartile increment was 1.039 [1.008-1.071]). Blood cadmium and lead were also significantly associated with subclinical carotid plaque. Bayesian Kernel Machine Regression analyses suggested a synergistic effect of copper-cadmium-lead mixture on subclinical carotid plaque presence. CONCLUSIONS Our findings identify copper as a novel risk factor of subclinical carotid atherosclerosis and show the potential synergistic proatherogenic effect of copper, cadmium, and lead mixture.
Collapse
Affiliation(s)
- Denglu Zhou
- Department of Cardiology, Institute of Cardiovascular Research, Xinqiao Hospital Army Medical University Chongqing China
| | - Qi Mao
- Department of Cardiology, Institute of Cardiovascular Research, Xinqiao Hospital Army Medical University Chongqing China
| | - Yapei Sun
- Center of Laboratory Medicine Chongqing Prevention and Treatment Center for Occupational Diseases Chongqing China
- Chongqing Key Laboratory of Prevention and Treatment for Occupational Diseases and Poisoning Chongqing China
- School of Public Health Nanjing Medical University Nanjing China
| | - Hao Cheng
- Department of Cardiology, Institute of Cardiovascular Research, Xinqiao Hospital Army Medical University Chongqing China
| | - Jianhua Zhao
- Department of Cardiology, Institute of Cardiovascular Research, Xinqiao Hospital Army Medical University Chongqing China
| | - Qingsong Liu
- Department of Cardiology, Institute of Cardiovascular Research, Xinqiao Hospital Army Medical University Chongqing China
| | - Mengyang Deng
- Department of Cardiology, Institute of Cardiovascular Research, Xinqiao Hospital Army Medical University Chongqing China
| | - Shangcheng Xu
- Center of Laboratory Medicine Chongqing Prevention and Treatment Center for Occupational Diseases Chongqing China
- Chongqing Key Laboratory of Prevention and Treatment for Occupational Diseases and Poisoning Chongqing China
- School of Public Health Nanjing Medical University Nanjing China
| | - Xiaohui Zhao
- Department of Cardiology, Institute of Cardiovascular Research, Xinqiao Hospital Army Medical University Chongqing China
| |
Collapse
|
13
|
Kirkham AM, Candeliere J, Fergusson D, Stelfox HT, Brandys T, McIsaac DI, Ramsay T, Roberts DJ. Prediction Models for Forecasting Risk of Development of Surgical Site Infection after Lower Limb Revascularization Surgery: A Systematic Review. Ann Vasc Surg 2024; 102:140-151. [PMID: 38307235 DOI: 10.1016/j.avsg.2023.11.034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Revised: 11/01/2023] [Accepted: 11/08/2023] [Indexed: 02/04/2024]
Abstract
BACKGROUND Surgical site infections (SSIs) are a common and potentially preventable complication of lower limb revascularization surgery associated with increased healthcare resource utilization and patient morbidity. We conducted a systematic review to evaluate multivariable prediction models designed to forecast risk of SSI development after these procedures. METHODS After protocol registration (CRD42022331292), we searched MEDLINE, EMBASE, CENTRAL, and Evidence-Based Medicine Reviews (inception to April 4th, 2023) for studies describing multivariable prediction models designed to forecast risk of SSI in adults after lower limb revascularization surgery. Two investigators independently screened abstracts and full-text articles, extracted data, and assessed risk of bias. A narrative synthesis was performed to summarize predictors included in the models and their calibration and discrimination, validation status, and clinical applicability. RESULTS Among the 6,671 citations identified, we included 5 studies (n = 23,063 patients). The included studies described 5 unique multivariable prediction models generated through forward selection, backward selection, or Akaike Information Criterion-based methods. Two models were designed to predict any SSI and 3 Szyilagyi grade II (extending into subcutaneous tissue) SSI. Across the 5 models, 18 adjusted predictors (10 of which were preoperative, 3 intraoperative, and 5 postoperative) significantly predicted any SSI and 14 adjusted predictors significantly predict Szilagyi grade II SSI. Female sex, obesity, and chronic obstructive pulmonary disease significantly predicted SSI in more than one model. All models had a "good fit" according to the Hosmer-Lemeshow test (P > 0.05). Model discrimination was quantified using the area under the curve, which ranged from 0.66 to 0.75 across models. Two models were internally validated using non-exhaustive twofold cross-validation and bootstrap resampling. No model was externally validated. Three studies had a high overall risk of bias according to the Prediction model Risk Of Bias ASsessment Tool (PROBAST). CONCLUSIONS Five multivariable prediction models with moderate discrimination have been developed to forecast risk of SSI development after lower limb revascularization surgery. Given the frequency and consequences of SSI after these procedures, development and external validation of novel prediction models and comparison of these models to the existing models evaluated in this systematic review is warranted.
Collapse
Affiliation(s)
- Aidan M Kirkham
- Division of Vascular and Endovascular Surgery, Department of Surgery, University of Ottawa, Ottawa, Ontario, Canada; Clinical Epidemiology Program, The Ottawa Hospital Research Institute, The Ottawa Hospital, Ottawa, Ontario, Canada; School of Epidemiology & Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Jasmine Candeliere
- Clinical Epidemiology Program, The Ottawa Hospital Research Institute, The Ottawa Hospital, Ottawa, Ontario, Canada
| | - Dean Fergusson
- Clinical Epidemiology Program, The Ottawa Hospital Research Institute, The Ottawa Hospital, Ottawa, Ontario, Canada; School of Epidemiology & Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Henry T Stelfox
- The O'Brien Institute for Public Health, University of Calgary, Calgary, Alberta, Canada; Departments of Critical Care Medicine, Medicine, and Community Health Sciences, University of Calgary, Calgary, Alberta, Canada
| | - Timothy Brandys
- Division of Vascular and Endovascular Surgery, Department of Surgery, University of Ottawa, Ottawa, Ontario, Canada
| | - Daniel I McIsaac
- Clinical Epidemiology Program, The Ottawa Hospital Research Institute, The Ottawa Hospital, Ottawa, Ontario, Canada; School of Epidemiology & Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada; Departments of Anesthesiology and Pain Medicine, University of Ottawa and The Ottawa Hospital, Ottawa, Ontario, Canada; Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada
| | - Tim Ramsay
- Clinical Epidemiology Program, The Ottawa Hospital Research Institute, The Ottawa Hospital, Ottawa, Ontario, Canada; School of Epidemiology & Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Derek J Roberts
- Division of Vascular and Endovascular Surgery, Department of Surgery, University of Ottawa, Ottawa, Ontario, Canada; Clinical Epidemiology Program, The Ottawa Hospital Research Institute, The Ottawa Hospital, Ottawa, Ontario, Canada; School of Epidemiology & Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada.
| |
Collapse
|
14
|
Zhang M, Kuo TT. Early prediction of long hospital stay for Intensive Care units readmission patients using medication information. Comput Biol Med 2024; 174:108451. [PMID: 38603899 DOI: 10.1016/j.compbiomed.2024.108451] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2023] [Revised: 03/21/2024] [Accepted: 04/07/2024] [Indexed: 04/13/2024]
Abstract
OBJECTIVE Predicting Intensive Care Unit (ICU) Length of Stay (LOS) accurately can improve patient wellness, hospital operations, and the health system's financial status. This study focuses on predicting the prolonged ICU LOS (≥3 days) of the 2nd admission, utilizing short historical data (1st admission only) for early-stage prediction, as well as incorporating medication information. MATERIALS AND METHODS We selected 18,572 ICU patients' records from the MIMIC-IV database for this study. We applied five machine learning classifiers: Logistic regression (LR), Random Forest (RF), Support Vector Machine (SVM), AdaBoost (AB) and XGBoost (XGB). We computed both the sum dose and the average dose for the medication and included them in our model. RESULTS The performance of the RF model demonstrates the highest level of accuracy compared to other models, as indicated by an Area Under the Receiver Operating Characteristic Curve (AUC) of 0.716 and an Expected Calibration Error (ECE) of 0.023. DISCUSSION The calibration improved all five classifiers (LR, RF, SVC, AB, XGB) in terms of ECE. The most important two features for RF are the length of 1st admission and the patient's age when they visited the hospital. The most important medication features are Phytonadione and Metoprolol Succinate XL. Also, both the sum and the average dose for the medication features contributed to the prediction task. CONCLUSION Our model showed the capability to predict the prolonged ICU LOS of the 2nd admission by utilizing the demographic, diagnosis, and medication information from the 1st admission. This method can potentially support the prevention of patient complications and enhance resource allocation in hospitals.
Collapse
Affiliation(s)
- Min Zhang
- Applied Statistics, University of Michigan, Ann Arbor, MI, 48109, USA
| | - Tsung-Ting Kuo
- UCSD Health Department of Biomedical Informatics, University of California San Diego, La Jolla, CA, 92093, USA.
| |
Collapse
|
15
|
Charu V, Liang JW, Mannalithara A, Kwong A, Tian L, Kim WR. Benchmarking clinical risk prediction algorithms with ensemble machine learning for the noninvasive diagnosis of liver fibrosis in NAFLD. Hepatology 2024:01515467-990000000-00869. [PMID: 38687634 DOI: 10.1097/hep.0000000000000908] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Accepted: 03/18/2024] [Indexed: 05/02/2024]
Abstract
BACKGROUND AND AIMS Ensemble machine-learning methods, like the superlearner, combine multiple models into a single one to enhance predictive accuracy. Here we explore the potential of the superlearner as a benchmarking tool for clinical risk prediction, illustrating the approach to identifying significant liver fibrosis among patients with NAFLD. APPROACH AND RESULTS We used 23 demographic/clinical variables to train superlearner(s) on data from the NASH-clinical research network observational study (n = 648) and validated models with data from the FLINT trial (n = 270) and National Health and Nutrition Examination Survey (NHANES) participants with NAFLD (n = 1244). Comparing the superlearner's performance to existing models (Fibrosis-4 [FIB-4], NAFLD fibrosis score, Forns, AST to Platelet Ratio Index [APRI], BARD, and Steatosis-Associated Fibrosis Estimator [SAFE]), it exhibited strong discriminative ability in the FLINT and NHANES validation sets, with AUCs of 0.79 (95% CI: 0.73-0.84) and 0.74 (95% CI: 0.68-0.79) respectively. CONCLUSIONS Notably, the SAFE score performed similarly to the superlearner, both of which outperformed FIB-4, APRI, Forns, and BARD scores in the validation data sets. Surprisingly, the superlearner derived from 12 base models matched the performance of one with 90 base models. Overall, the superlearner, being the "best-in-class" machine-learning predictor, excelled in detecting fibrotic NASH, and this approach can be used to benchmark the performance of conventional clinical risk prediction models.
Collapse
Affiliation(s)
- Vivek Charu
- Quantitative Sciences Unit, Department of Medicine, Stanford University School of Medicine, Stanford, California, USA
- Department of Pathology, Stanford University School of Medicine, Stanford, California, USA
| | - Jane W Liang
- Quantitative Sciences Unit, Department of Medicine, Stanford University School of Medicine, Stanford, California, USA
| | - Ajitha Mannalithara
- Department of Medicine, Division of Gastroenterology and Hepatology, Stanford University School of Medicine, Stanford, California, USA
| | - Allison Kwong
- Department of Medicine, Division of Gastroenterology and Hepatology, Stanford University School of Medicine, Stanford, California, USA
| | - Lu Tian
- Department of Biomedical Data Sciences, Stanford University School of Medicine, Stanford, California, USA
| | - W Ray Kim
- Department of Medicine, Division of Gastroenterology and Hepatology, Stanford University School of Medicine, Stanford, California, USA
| |
Collapse
|
16
|
Nie GL, Yan J, Li Y, Zhang HL, Xie DN, Zhu XW, Li X. Predictive model for non-malignant portal vein thrombosis associated with cirrhosis based on inflammatory biomarkers. World J Gastrointest Oncol 2024; 16:1213-1226. [PMID: 38660630 PMCID: PMC11037040 DOI: 10.4251/wjgo.v16.i4.1213] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/08/2023] [Revised: 01/18/2024] [Accepted: 02/23/2024] [Indexed: 04/10/2024] Open
Abstract
BACKGROUND Portal vein thrombosis (PVT), a complication of liver cirrhosis, is a major public health concern. PVT prediction is the most effective method for PVT diagnosis and treatment. AIM To develop and validate a nomogram and network calculator based on clinical indicators to predict PVT in patients with cirrhosis. METHODS Patients with cirrhosis hospitalized between January 2016 and December 2021 at the First Hospital of Lanzhou University were screened and 643 patients with cirrhosis who met the eligibility criteria were retrieved. Following a 1:1 propensity score matching 572 patients with cirrhosis were screened, and relevant clinical data were collected. PVT risk factors were identified using the least absolute shrinkage and selection operator (LASSO) and multivariate logistic regression analysis. Variance inflation factors and correlation matrix plots were used to analyze multicollinearity among the variables. A nomogram was constructed to predict the probability of PVT based on independent risk factors for PVT, and its predictive performance was verified using a receiver operating characteristic curve (ROC), calibration curves, and decision curve analysis (DCA). Finally, a network calculator was constructed based on the nomograms. RESULTS This study enrolled 286 cirrhosis patients with PVT and 286 without PVT. LASSO analysis revealed 13 variables as strongly associated with PVT occurrence. Multivariate logistic regression analysis revealed nine indicators as independent PVT risk factors, including etiology, ascites, gastroesophageal varices, platelet count, D-dimer, portal vein diameter, portal vein velocity, aspartate transaminase to neutrophil ratio index, and platelet-to-lymphocyte ratio. LASSO and correlation matrix plot results revealed no significant multicollinearity or correlation among the variables. A nomogram was constructed based on the screened independent risk factors. The nomogram had excellent predictive performance, with an area under the ROC curve of 0.821 and 0.829 in the training and testing groups, respectively. Calibration curves and DCA revealed its good clinical performance. Finally, the optimal cutoff value for the total nomogram score was 0.513. The sensitivity and specificity of the optimal cutoff values were 0.822 and 0.706, respectively. CONCLUSION A nomogram for predicting PVT occurrence was successfully developed and validated, and a network calculator was constructed. This can enable clinicians to rapidly and easily identify high PVT risk groups.
Collapse
Affiliation(s)
- Guo-Le Nie
- The First School of Clinical Medicine, Lanzhou University, Lanzhou 730000, Gansu Province, China
| | - Jun Yan
- Department of General Surgery, The First Hospital of Lanzhou University, Lanzhou 730000, Gansu Province, China
| | - Ying Li
- Department of General Surgery, The First Hospital of Lanzhou University, Lanzhou 730000, Gansu Province, China
| | - Hong-Long Zhang
- The First School of Clinical Medicine, Lanzhou University, Lanzhou 730000, Gansu Province, China
| | - Dan-Na Xie
- The First School of Clinical Medicine, Lanzhou University, Lanzhou 730000, Gansu Province, China
| | - Xing-Wang Zhu
- The First School of Clinical Medicine, Lanzhou University, Lanzhou 730000, Gansu Province, China
| | - Xun Li
- Department of General Surgery, The First Hospital of Lanzhou University, Lanzhou 730000, Gansu Province, China
| |
Collapse
|
17
|
Pan Y, Xie F, Zeng W, Chen H, Chen Z, Xu D, Chen Y. T cell-mediated tumor killing sensitivity gene signature-based prognostic score for acute myeloid leukemia. Discov Oncol 2024; 15:121. [PMID: 38619693 PMCID: PMC11018597 DOI: 10.1007/s12672-024-00962-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/11/2024] [Accepted: 03/29/2024] [Indexed: 04/16/2024] Open
Abstract
BACKGROUND AND OBJECTIVE Acute myeloid leukemia (AML) is an aggressive, heterogenous hematopoetic malignancies with poor long-term prognosis. T-cell mediated tumor killing plays a key role in tumor immunity. Here, we explored the prognostic performance and functional significance of a T-cell mediated tumor killing sensitivity gene (GSTTK)-based prognostic score (TTKPI). METHODS Publicly available transcriptomic data for AML were obtained from TCGA and NCBI-GEO. GSTTK were identified from the TISIDB database. Signature GSTTK for AML were identified by differential expression analysis, COX proportional hazards and LASSO regression analysis and a comprehensive TTKPI score was constructed. Prognostic performance of the TTKPI was examined using Kaplan-Meier survival analysis, Receiver operating curves, and nomogram analysis. Association of TTKPI with clinical phenotypes, tumor immune cell infiltration patterns, checkpoint expression patterns were analysed. Drug docking was used to identify important candidate drugs based on the TTKPI-component genes. RESULTS From 401 differentially expressed GSTTK in AML, 24 genes were identified as signature genes and used to construct the TTKPI score. High-TTKPI risk score predicted worse survival and good prognostic accuracy with AUC values ranging from 75 to 96%. Higher TTKPI scores were associated with older age and cancer stage, which showed improved prognostic performance when combined with TTKPI. High TTKPI was associated with lower naïve CD4 T cell and follicular helper T cell infiltrates and higher M2 macrophages/monocyte infiltration. Distinct patterns of immune checkpoint expression corresponded with TTKPI score groups. Three agents; DB11791 (Capmatinib), DB12886 (GSK-1521498) and DB14773 (Lifirafenib) were identified as candidates for AML. CONCLUSION A T-cell mediated killing sensitivity gene-based prognostic score TTKPI showed good accuracy in predicting survival in AML. TTKPI corresponded to functional and immunological features of the tumor microenvironment including checkpoint expression patterns and should be investigated for precision medicine approaches.
Collapse
Affiliation(s)
- Yiyun Pan
- Suzhou Medical College of Soochow University, Suzhou, 215123, Jiangsu, People's Republic of China
- Ganzhou Cancer Hospital, Gannan Medical University, No.19, Huayuan Road, Zhanggong Avenue, Ganzhou, Jiangxi, People's Republic of China
| | - FangFang Xie
- Ganzhou People's Hospital, Ganzhou, 341000, Jiangxi, People's Republic of China
| | - Wen Zeng
- Ganzhou Cancer Hospital, Gannan Medical University, No.19, Huayuan Road, Zhanggong Avenue, Ganzhou, Jiangxi, People's Republic of China
| | - Hailong Chen
- Ganzhou Cancer Hospital, Gannan Medical University, No.19, Huayuan Road, Zhanggong Avenue, Ganzhou, Jiangxi, People's Republic of China
| | - Zhengcong Chen
- Ganzhou Cancer Hospital, Gannan Medical University, No.19, Huayuan Road, Zhanggong Avenue, Ganzhou, Jiangxi, People's Republic of China
| | - Dechang Xu
- Ganzhou Cancer Hospital, Gannan Medical University, No.19, Huayuan Road, Zhanggong Avenue, Ganzhou, Jiangxi, People's Republic of China.
| | - Yijian Chen
- Suzhou Medical College of Soochow University, Suzhou, 215123, Jiangsu, People's Republic of China.
- The First Affiliated Hospital of Gannan Medical University, No.23, Qingnian Road, Zhanggong Avenue, Ganzhou, 8105640, Jiangxi, People's Republic of China.
| |
Collapse
|
18
|
Truong B, Zheng J, Hornsby L, Fox B, Chou C, Qian J. Development and Validation of Machine Learning Algorithms to Predict 1-Year Ischemic Stroke and Bleeding Events in Patients with Atrial Fibrillation and Cancer. Cardiovasc Toxicol 2024; 24:365-374. [PMID: 38499940 PMCID: PMC10998799 DOI: 10.1007/s12012-024-09843-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/15/2024] [Accepted: 02/21/2024] [Indexed: 03/20/2024]
Abstract
In this study, we leveraged machine learning (ML) approach to develop and validate new assessment tools for predicting stroke and bleeding among patients with atrial fibrillation (AFib) and cancer. We conducted a retrospective cohort study including patients who were newly diagnosed with AFib with a record of cancer from the 2012-2018 Surveillance, Epidemiology, and End Results (SEER)-Medicare database. The ML algorithms were developed and validated separately for each outcome by fitting elastic net, random forest (RF), extreme gradient boosting (XGBoost), support vector machine (SVM), and neural network models with tenfold cross-validation (train:test = 7:3). We obtained area under the curve (AUC), sensitivity, specificity, and F2 score as performance metrics. Model calibration was assessed using Brier score. In sensitivity analysis, we resampled data using Synthetic Minority Oversampling Technique (SMOTE). Among 18,388 patients with AFib and cancer, 523 (2.84%) had ischemic stroke and 221 (1.20%) had major bleeding within one year after AFib diagnosis. In prediction of ischemic stroke, RF significantly outperformed other ML models [AUC (0.916, 95% CI 0.887-0.945), sensitivity 0.868, specificity 0.801, F2 score 0.375, Brier score = 0.035]. However, the performance of ML algorithms in prediction of major bleeding was low with highest AUC achieved by RF (0.623, 95% CI 0.554-0.692). RF models performed better than CHA2DS2-VASc and HAS-BLED scores. SMOTE did not improve the performance of the ML algorithms. Our study demonstrated a promising application of ML in stroke prediction among patients with AFib and cancer. This tool may be leveraged in assisting clinicians to identify patients at high risk of stroke and optimize treatment decisions.
Collapse
Affiliation(s)
- Bang Truong
- Department of Health Outcomes Research and Policy, Auburn University Harrison College of Pharmacy, 4306d Walker Building, Auburn, AL, 36849, USA
| | - Jingyi Zheng
- Department of Mathematics and Statistics, Auburn University College of Sciences and Mathematics, Auburn, AL, USA
| | - Lori Hornsby
- Department of Pharmacy Practice, Auburn University Harrison College of Pharmacy, Auburn, AL, USA
| | - Brent Fox
- Department of Health Outcomes Research and Policy, Auburn University Harrison College of Pharmacy, 4306d Walker Building, Auburn, AL, 36849, USA
| | - Chiahung Chou
- Department of Health Outcomes Research and Policy, Auburn University Harrison College of Pharmacy, 4306d Walker Building, Auburn, AL, 36849, USA
| | - Jingjing Qian
- Department of Health Outcomes Research and Policy, Auburn University Harrison College of Pharmacy, 4306d Walker Building, Auburn, AL, 36849, USA.
| |
Collapse
|
19
|
Rastogi R, Okunowo O, Faerber JA, Mavroudis CD, Whitworth H, Giglia TM, Witmer C, Raffini LJ, O'Byrne ML. Incidence, Management, and Outcomes of Pulmonary Embolism at Tertiary Pediatric Hospitals in the United States. JACC. ADVANCES 2024; 3:100895. [PMID: 38939674 PMCID: PMC11198360 DOI: 10.1016/j.jacadv.2024.100895] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Revised: 12/18/2023] [Accepted: 01/02/2024] [Indexed: 06/29/2024]
Abstract
Background Pediatric pulmonary embolism (PE) is rare and potentially life-threatening. Though thrombolysis and thrombectomy are increasingly used in adult PE, trends in pediatric treatment and outcomes remain incompletely described. Objectives The purpose of this study was to describe the incidence of PE, proportion of cases treated with anticoagulation alone, systemic thrombolysis, and directed therapy (local thrombolysis and thrombectomy), clinical outcomes, and total costs. Methods A multicenter observational study was performed using administrative data from the Pediatric Health Information System database to study PE treated at U.S. pediatric hospitals from 2015 to 2021. Outcomes by treatment were evaluated using multivariable generalized linear mixed effects models. Results Of 3,136 subjects, 70% were at least 12 years of age, and 46% were male. Sixty-two percent had at least 1 comorbidity, and congenital heart disease of any kind was the most prevalent (20%). Eighty-eight percent of subjects received anticoagulation alone, 7% received systemic thrombolysis, and 5% received directed therapy. Overall in-hospital mortality was 7.5%. Treatment approach did not change over time (P = 0.98). After adjusting for patient characteristics, directed therapy was associated with a lower risk of mortality (adjusted percentage -3%, [95% CI: -5% to 0%]) than anticoagulation alone. Systemic thrombolysis was associated with a greater total cost of hospitalization ($113,043 greater [95% CI: $62,866, $163,219]). Length of hospital stay did not differ by treatment. Conclusions Pediatric patients with PE have a high incidence of underlying chronic disease. Anticoagulation alone remains the mainstay of treatment, with thrombolysis and thrombectomy rarely being used. Given the relative rarity of pediatric PE, additional research requiring innovative study designs is paramount.
Collapse
Affiliation(s)
- Radhika Rastogi
- Department of Pediatrics, The Children’s Hospital of Philadelphia and Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Oluwatimilehin Okunowo
- Division of Biostatistics, Department of Computational and Quantitative Medicine, Beckman Research Institute of City of Hope, Duarte, California, USA
| | - Jennifer A. Faerber
- Data Science and Biostatistics Unit, Department of Biomedical and Health Informatics, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| | - Constantine D. Mavroudis
- Division of Cardiothoracic Surgery, Department of Surgery, The Children’s Hospital of Philadelphia, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Hilary Whitworth
- Division of Hematology, Department of Pediatrics, The Children’s Hospital of Philadelphia, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Therese M. Giglia
- Division of Cardiology, Department of Pediatrics, The Children’s Hospital of Philadelphia, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Char Witmer
- Division of Hematology, Department of Pediatrics, The Children’s Hospital of Philadelphia, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Leslie J. Raffini
- Division of Hematology, Department of Pediatrics, The Children’s Hospital of Philadelphia, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Michael L. O'Byrne
- Division of Cardiology, Department of Pediatrics, The Children’s Hospital of Philadelphia, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Clinical Futures, Department of Pediatrics, The Children’s Hospital of Philadelphia, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Leonard Davis Institute and Cardiovascular Outcomes, Quality, and Evaluative Research Center, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania, USA
| |
Collapse
|
20
|
Xia M, Jin C, Zheng Y, Wang J, Zhao M, Cao S, Xu T, Pei B, Irwin MG, Lin Z, Jiang H. Deep learning-based facial analysis for predicting difficult videolaryngoscopy: a feasibility study. Anaesthesia 2024; 79:399-409. [PMID: 38093485 DOI: 10.1111/anae.16194] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/03/2023] [Indexed: 03/07/2024]
Abstract
While videolaryngoscopy has resulted in better overall success rates of tracheal intubation, airway assessment is still an important prerequisite for safe airway management. This study aimed to create an artificial intelligence model to identify difficult videolaryngoscopy using a neural network. Baseline characteristics, medical history, bedside examination and seven facial images were included as predictor variables. ResNet-18 was introduced to recognise images and extract features. Different machine learning algorithms were utilised to develop predictive models. A videolaryngoscopy view of Cormack-Lehane grade of 1 or 2 was classified as 'non-difficult', while grade 3 or 4 was classified as 'difficult'. A total of 5849 patients were included, of whom 5335 had non-difficult and 514 had difficult videolaryngoscopy. The facial model (only including facial images) using the Light Gradient Boosting Machine algorithm showed the highest area under the curve (95%CI) of 0.779 (0.733-0.825) with a sensitivity (95%CI) of 0.757 (0.650-0.845) and specificity (95%CI) of 0.721 (0.626-0.794) in the test set. Compared with bedside examination and multivariate scores (El-Ganzouri and Wilson), the facial model had significantly higher predictive performance (p < 0.001). Artificial intelligence-based facial analysis is a feasible technique for predicting difficulty during videolaryngoscopy, and the model developed using neural networks has higher predictive performance than traditional methods.
Collapse
Affiliation(s)
- M Xia
- Department of Anaesthesiology, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - C Jin
- Department of Anaesthesiology, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Y Zheng
- State Key Laboratory of Ocean Engineering, School of Naval Architecture, Ocean and Civil Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - J Wang
- Department of Anaesthesiology, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - M Zhao
- State Key Laboratory of Ocean Engineering, School of Naval Architecture, Ocean and Civil Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - S Cao
- Department of Anaesthesiology, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - T Xu
- Department of Anaesthesiology, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - B Pei
- Department of Anaesthesiology, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - M G Irwin
- Department of Anaesthesiology, University of Hong Kong, Hong Kong
| | - Z Lin
- State Key Laboratory of Ocean Engineering, School of Naval Architecture, Ocean and Civil Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - H Jiang
- Department of Anaesthesiology, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| |
Collapse
|
21
|
Cheng YW, Kuo PC, Chen SH, Kuo YT, Liu TL, Chan WS, Chan KC, Yeh YC. Early prediction of mortality at sepsis diagnosis time in critically ill patients by using interpretable machine learning. J Clin Monit Comput 2024; 38:271-279. [PMID: 38150124 DOI: 10.1007/s10877-023-01108-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2023] [Accepted: 11/15/2023] [Indexed: 12/28/2023]
Abstract
This study applied machine learning for the early prediction of 30-day mortality at sepsis diagnosis time in critically ill patients. Retrospective study using data collected from the Medical Information Mart for Intensive Care IV database. The data of the patient cohort was divided on the basis of the year of hospitalization, into training (2008-2013), validation (2014-2016), and testing (2017-2019) datasets. 24,377 patients with the sepsis diagnosis time < 24 h after intensive care unit (ICU) admission were included. A gradient boosting tree-based algorithm (XGBoost) was used for training the machine learning model to predict 30-day mortality at sepsis diagnosis time in critically ill patients. Model performance was measured in both discrimination and calibration aspects. The model was interpreted using the SHapley Additive exPlanations (SHAP) module. The 30-day mortality rate of the testing dataset was 17.9%, and 39 features were selected for the machine learning model. Model performance on the testing dataset achieved an area under the receiver operating characteristic curve (AUROC) of 0.853 (95% CI 0.837-0.868) and an area under the precision-recall curves of 0.581 (95% CI 0.541-0.619). The calibration plot for the model revealed a slope of 1.03 (95% CI 0.94-1.12) and intercept of 0.14 (95% CI 0.04-0.25). The SHAP revealed the top three most significant features, namely age, increased red blood cell distribution width, and respiratory rate. Our study demonstrated the feasibility of using the interpretable machine learning model to predict mortality at sepsis diagnosis time.
Collapse
Affiliation(s)
- Yi-Wei Cheng
- Taiwan AI Labs, Taipei, Taiwan
- Department of Anesthesiology, National Taiwan University Hospital, No. 7, Chung Shan South Road, Taipei, Taiwan
| | - Po-Chih Kuo
- Department of Computer Science, National Tsing Hua University, Hsinchu, Taiwan
| | - Shih-Hong Chen
- Department of Anesthesiology, Taipei Tzu Chi Hospital, New Taipei, Taiwan
| | - Yu-Ting Kuo
- Department of Anesthesiology, National Taiwan University Hospital, No. 7, Chung Shan South Road, Taipei, Taiwan
| | | | - Wing-Sum Chan
- Department of Anesthesiology, Far Eastern Memorial Hospital, No. 21, Section 2, Nanya S Rd, Banqiao District, New Taipei City, 220, Taiwan.
| | - Kuang-Cheng Chan
- Department of Anesthesiology, National Taiwan University Hospital, No. 7, Chung Shan South Road, Taipei, Taiwan
| | - Yu-Chang Yeh
- Department of Anesthesiology, National Taiwan University Hospital, No. 7, Chung Shan South Road, Taipei, Taiwan.
| |
Collapse
|
22
|
Lapi F, Marconi E, Domnich A, Cricelli I, Rossi A, Grattagliano I, Icardi G, Cricelli C. A Vulnerability Index to Assess the Risk of SARS-CoV-2-Related Hospitalization/Death: Urgent Need for an Update after Diffusion of Anti-COVID Vaccines. Infect Dis Rep 2024; 16:260-268. [PMID: 38525768 PMCID: PMC10961815 DOI: 10.3390/idr16020021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/15/2023] [Revised: 03/11/2024] [Accepted: 03/12/2024] [Indexed: 03/26/2024] Open
Abstract
Background: There are algorithms to predict the risk of SARS-CoV-2-related complications. Given the spread of anti-COVID vaccination, which sensibly modified the burden of risk of the infection, these tools need to be re-calibrated. Therefore, we updated our vulnerability index, namely, the Health Search (HS)-CoVulnerabiltyIndex (VI)d (HS-CoVId), to predict the risk of SARS-CoV-2-related hospitalization/death in the primary care setting. Methods: We formed a cohort of individuals aged ≥15 years and diagnosed with COVID-19 between 1 January and 31 December 2021 in the HSD. The date of COVID-19 diagnosis was the study index date. These patients were eligible if they had received an anti-COVID vaccine at least 15 days before the index date. Patients were followed up from the index date until one of the following events, whichever came first: COVID-19-related hospitalization/death (event date), end of registration with their GPs, and end of the study period (31 December 2022). To calculate the incidence rate of COVID-19-related hospitalization/death, a patient-specific score was derived through linear combination of the coefficients stemming from a multivariate Cox regression model. Its prediction performance was evaluated by obtaining explained variation, discrimination, and calibration measures. Results: We identified 2192 patients who had received an anti-COVID vaccine from 1 January to 31 December 2021. With this cohort, we re-calibrated the HS-CoVId by calculating optimism-corrected pseudo-R2, AUC, and calibration slope. The final model reported a good predictive performance by explaining 58% (95% CI: 48-71%) of variation in the occurrence of hospitalizations/deaths, the AUC was 83 (95% CI: 77-93%), and the calibration slope did not reject the equivalence hypothesis (p-value = 0.904). Conclusions: Two versions of HS-CoVId need to be differentially adopted to assess the risk of COVID-19-related complications among vaccinated and unvaccinated subjects. Therefore, this functionality should be operationalized in related patient- and population-based informatic tools intended for general practitioners.
Collapse
Affiliation(s)
- Francesco Lapi
- Health Search, Italian College of General Practitioners and Primary Care, 50142 Florence, Italy
| | - Ettore Marconi
- Health Search, Italian College of General Practitioners and Primary Care, 50142 Florence, Italy
| | - Alexander Domnich
- Hygiene Unit, San Martino Policlinico Hospital-IRCCS for Oncology and Neurosciences, 16132 Genoa, Italy; (A.D.); (G.I.)
| | | | - Alessandro Rossi
- Italian College of General Practitioners and Primary Care, 50142 Florence, Italy; (A.R.); (I.G.); (C.C.)
| | - Ignazio Grattagliano
- Italian College of General Practitioners and Primary Care, 50142 Florence, Italy; (A.R.); (I.G.); (C.C.)
| | - Giancarlo Icardi
- Hygiene Unit, San Martino Policlinico Hospital-IRCCS for Oncology and Neurosciences, 16132 Genoa, Italy; (A.D.); (G.I.)
- Department of Health Sciences, University of Genoa, 16132 Genoa, Italy
| | - Claudio Cricelli
- Italian College of General Practitioners and Primary Care, 50142 Florence, Italy; (A.R.); (I.G.); (C.C.)
| |
Collapse
|
23
|
Iantorno SE, Scaife JH, Bryce JR, Yang M, McCrum ML, Bucher BT. Emergency Department Utilization for Pediatric Gastrostomy Tubes Across the United States. J Surg Res 2024; 295:820-826. [PMID: 38160493 DOI: 10.1016/j.jss.2023.11.028] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Revised: 09/30/2023] [Accepted: 11/12/2023] [Indexed: 01/03/2024]
Abstract
INTRODUCTION Emergency Department (ED) visits for gastrostomy tube complications in children represent a substantial health-care burden, and many ED visits are potentially preventable. The number and nature of ED visits to community hospitals for pediatric gastrostomy tube complications is unknown. METHODS Using the 2019 Nationwide Emergency Department Sample, we performed a retrospective cross-sectional analysis of pediatric patients (<18 y) with a primary diagnosis of gastrostomy tube complication. Our primary outcome was a potentially preventable ED visit, defined as an encounter that did not result in any imaging, procedures, or an inpatient admission. Univariate and multivariable logistic regression analyses were used to determine the associations between patient factors and our primary outcome. RESULTS We observed 32,036 ED visits at 535 hospitals and 15,165 (47.3%) were potentially preventable. The median (interquartile range) age was 2 (1, 6) years and 17,707 (55%) were male. Compared to White patients, patients with higher odds of potentially preventable visits were Black (adjusted odds ratio (aOR) [95% confidence interval {CI}]: 1.07 [1.05-1.11], P < 0.001) and Hispanic (aOR [95% CI]: 1.05 [1.02-1.08], P = 0.004). Patients with residential zip codes in the first (aOR [95% CI]: 1.08 [1.04, 1.12], P < 0.001), second (aOR [95% CI]: 1.07 [1.03, 1.11], P < 0.001), and third (aOR [95% CI]: 1.09 [1.05, 1.13], P < 0.001) median household income quartiles had higher odds of potentially preventable visits compared to the highest. CONCLUSIONS In a nationally representative sample of EDs, 47.3% of visits for pediatric gastrostomy tubes were potentially preventable. Efforts to improve outpatient management are warranted to reduce health-care utilization for these patients.
Collapse
Affiliation(s)
- Stephanie E Iantorno
- Department of Surgery, University of Utah School of Medicine, Salt Lake City, Utah.
| | - Jack H Scaife
- Department of Surgery, University of Utah School of Medicine, Salt Lake City, Utah
| | - Jacoby R Bryce
- Department of Surgery, University of Utah School of Medicine, Salt Lake City, Utah
| | - Meng Yang
- Department of Surgery, University of Utah School of Medicine, Salt Lake City, Utah
| | - Marta L McCrum
- Department of Surgery, University of Utah School of Medicine, Salt Lake City, Utah
| | - Brian T Bucher
- Department of Surgery, University of Utah School of Medicine, Salt Lake City, Utah
| |
Collapse
|
24
|
Monterrubio-Gómez K, Constantine-Cooke N, Vallejos CA. A review on statistical and machine learning competing risks methods. Biom J 2024; 66:e2300060. [PMID: 38351217 DOI: 10.1002/bimj.202300060] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2023] [Revised: 08/31/2023] [Accepted: 10/15/2023] [Indexed: 02/16/2024]
Abstract
When modeling competing risks (CR) survival data, several techniques have been proposed in both the statistical and machine learning literature. State-of-the-art methods have extended classical approaches with more flexible assumptions that can improve predictive performance, allow high-dimensional data and missing values, among others. Despite this, modern approaches have not been widely employed in applied settings. This article aims to aid the uptake of such methods by providing a condensed compendium of CR survival methods with a unified notation and interpretation across approaches. We highlight available software and, when possible, demonstrate their usage via reproducible R vignettes. Moreover, we discuss two major concerns that can affect benchmark studies in this context: the choice of performance metrics and reproducibility.
Collapse
Affiliation(s)
| | - Nathan Constantine-Cooke
- MRC Human Genetics Unit, University of Edinburgh, Edinburgh, UK
- Centre for Genomic and Experimental Medicine, Institute of Genetics and Cancer, University of Edinburgh, Edinburgh, UK
| | - Catalina A Vallejos
- MRC Human Genetics Unit, University of Edinburgh, Edinburgh, UK
- The Alan Turing Institute, London, UK
| |
Collapse
|
25
|
Tian T, Hu W, Hao J. Nomogram for predicting neutropenia in patients with esophageal, gastric, or colorectal cancer treated by chemotherapy in the first cycle. Int J Biol Markers 2024; 39:23-30. [PMID: 38291662 DOI: 10.1177/03936155241228304] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/01/2024]
Abstract
OBJECTIVES Development and validation of a predictive model including serum vitamin concentration to estimate the risk of chemotherapy-induced grade 3/4 neutropenia in esophageal cancer, gastric cancer, or colorectal cancer patients who receive the first cycle of chemotherapy. METHODS Data from 535 patients treated at the Affiliated Fuyang People's Hospital of Anhui Medical University from January 1, 2020, to March 2, 2022, were used to derive the predictive model. Least absolute shrinkage and selection operator regression analysis was performed to screen potential risk characteristics, and multivariate logistic regression was utilized to investigate efficient factors associated with chemotherapy-induced neutropenia. A nomogram was constructed using this logistic model. This nomogram was then tested on a temporal validation cohort containing 212 consecutive patients. RESULTS In the cohort of all 747 eligible patients, grade 3/4 neutropenia incidence was 45.2%. Age, Eastern Cooperative Oncology Group-performance status, neutrophil count, serum albumin, and hemoglobin data were entered into the final model. The performance of the final predictive nomogram was assessed by the area under the receiver operating characteristic curve in both the development and validation datasets. The calibration curves indicated that the estimated risks were accurate. Decision curve analysis for the predictive model exhibited improved clinical practicality. CONCLUSION In the present study, we established an accessible risk predictive model and identified valuable serum vitamin concentration parameters associated with chemotherapy-induced neutropenia. The predictive model may improve the grade 3/4 neutropenia risk prediction in patients with gastrointestinal malignancies who receive oxaliplatin- and fluoropyrimidine-based chemotherapy and help physicians make appropriate decisions for disease management.
Collapse
Affiliation(s)
- Tian Tian
- Department of Oncology, First Affiliated Hospital of Anhui Medical University, Hefei, China
- Department of Oncology, Affiliated Fuyang People's Hospital of Anhui Medical University (Fuyang People's Hospital), Fuyang, China
| | - Wenjun Hu
- Department of Oncology, Affiliated Fuyang People's Hospital of Anhui Medical University (Fuyang People's Hospital), Fuyang, China
| | - Jiqing Hao
- Department of Oncology, First Affiliated Hospital of Anhui Medical University, Hefei, China
| |
Collapse
|
26
|
Adeoye J, Su YX. Leveraging artificial intelligence for perioperative cancer risk assessment of oral potentially malignant disorders. Int J Surg 2024; 110:1677-1686. [PMID: 38051932 PMCID: PMC10942172 DOI: 10.1097/js9.0000000000000979] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2023] [Accepted: 11/21/2023] [Indexed: 12/07/2023]
Abstract
Oral potentially malignant disorders (OPMDs) are mucosal conditions with an inherent disposition to develop oral squamous cell carcinoma. Surgical management is the most preferred strategy to prevent malignant transformation in OPMDs, and surgical approaches to treatment include conventional scalpel excision, laser surgery, cryotherapy, and photodynamic therapy. However, in reality, since all patients with OPMDs will not develop oral squamous cell carcinoma in their lifetime, there is a need to stratify patients according to their risk of malignant transformation to streamline surgical intervention for patients with the highest risks. Artificial intelligence (AI) has the potential to integrate disparate factors influencing malignant transformation for robust, precise, and personalized cancer risk stratification of OPMD patients than current methods to determine the need for surgical resection, excision, or re-excision. Therefore, this article overviews existing AI models and tools, presents a clinical implementation pathway, and discusses necessary refinements to aid the clinical application of AI-based platforms for cancer risk stratification of OPMDs in surgical practice.
Collapse
Affiliation(s)
| | - Yu-Xiong Su
- Division of Oral and Maxillofacial Surgery, Faculty of Dentistry, University of Hong Kong, Hong Kong SAR, People’s Republic of China
| |
Collapse
|
27
|
Tan J, Liu C, Yang M, Xiong Y, Huang S, Qi Y, Chen M, Thabane L, Liu X, He L, Sun X. Investigation of statistical methods used in prognostic prediction models for obstetric care: A 10 year-span cross-sectional study. Acta Obstet Gynecol Scand 2024; 103:611-620. [PMID: 38140844 PMCID: PMC10867372 DOI: 10.1111/aogs.14757] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2023] [Revised: 11/06/2023] [Accepted: 12/06/2023] [Indexed: 12/24/2023]
Abstract
INTRODUCTION Obstetric care is a highly active area in the development and application of prognostic prediction models. The development and validation of these models often require the utilization of advanced statistical techniques. However, failure to adhere to rigorous methodological standards could greatly undermine the reliability and trustworthiness of the resultant models. Consequently, the aim of our study was to examine the current statistical practices employed in obstetric care and offer recommendations to enhance the utilization of statistical methods in the development of prognostic prediction models. MATERIAL AND METHODS We conducted a cross-sectional survey using a sample of studies developing or validating prognostic prediction models for obstetric care published in a 10-year span (2011-2020). A structured questionnaire was developed to investigate the statistical issues in five domains, including model derivation (predictor selection and algorithm development), model validation (internal and external), model performance, model presentation, and risk threshold setting. On the ground of survey results and existing guidelines, a list of recommendations for statistical methods in prognostic models was developed. RESULTS A total of 112 eligible studies were included, with 107 reporting model development and five exclusively reporting external validation. During model development, 58.9% of the studies did not include any form of validation. Of these, 46.4% used stepwise regression in a crude manner for predictor selection, while two-thirds made decisions on retaining or dropping candidate predictors solely based on p-values. Additionally, 26.2% transformed continuous predictors into categorical variables, and 80.4% did not consider nonlinear relationships between predictors and outcomes. Surprisingly, 94.4% of the studies did not examine the correlation between predictors. Moreover, 47.1% of the studies did not compare population characteristics between the development and external validation datasets, and only one-fifth evaluated both discrimination and calibration. Furthermore, 53.6% of the studies did not clearly present the model, and less than half established a risk threshold to define risk categories. In light of these findings, 10 recommendations were formulated to promote the appropriate use of statistical methods. CONCLUSIONS The use of statistical methods is not yet optimal. Ten recommendations were offered to assist the statistical methods of prognostic prediction models in obstetric care.
Collapse
Affiliation(s)
- Jing Tan
- Chinese Evidence‐based Medicine Center, West China HospitalSichuan UniversityChengduSichuanChina
- NMPA Key Laboratory for Real World Data Research and Evaluation in HainanChengduSichuanChina
- Department of Health Research Methods, Evidence, and ImpactMcMaster UniversityHamiltonOntarioCanada
- Biostatistics UnitSt Joseph's Healthcare—HamiltonHamiltonOntarioCanada
| | - Chunrong Liu
- Chinese Evidence‐based Medicine Center, West China HospitalSichuan UniversityChengduSichuanChina
- NMPA Key Laboratory for Real World Data Research and Evaluation in HainanChengduSichuanChina
| | - Min Yang
- Department of Epidemiology and Biostatistics, West China School of Public HealthSichuan UniversityChengduChina
- Faculty of Health, Design and ArtSwinburne Technology UniversityMelbourneVictoriaAustralia
| | - Yiquan Xiong
- Chinese Evidence‐based Medicine Center, West China HospitalSichuan UniversityChengduSichuanChina
- NMPA Key Laboratory for Real World Data Research and Evaluation in HainanChengduSichuanChina
| | - Shiyao Huang
- Chinese Evidence‐based Medicine Center, West China HospitalSichuan UniversityChengduSichuanChina
- NMPA Key Laboratory for Real World Data Research and Evaluation in HainanChengduSichuanChina
| | - Yana Qi
- Chinese Evidence‐based Medicine Center, West China HospitalSichuan UniversityChengduSichuanChina
- NMPA Key Laboratory for Real World Data Research and Evaluation in HainanChengduSichuanChina
| | - Meng Chen
- Department of Obstetrics and Gynecology, and Key Laboratory of Birth Defects and Related Diseases of Women and Children (Sichuan University), Ministry of Education, West China Second University HospitalSichuan UniversityChengduSichuanChina
| | - Lehana Thabane
- Department of Health Research Methods, Evidence, and ImpactMcMaster UniversityHamiltonOntarioCanada
- Biostatistics UnitSt Joseph's Healthcare—HamiltonHamiltonOntarioCanada
| | - Xinghui Liu
- Department of Obstetrics and Gynecology, and Key Laboratory of Birth Defects and Related Diseases of Women and Children (Sichuan University), Ministry of Education, West China Second University HospitalSichuan UniversityChengduSichuanChina
| | - Lin He
- The Intelligence Library Center, Ministry of Science and Technology, Chinese Evidence‐Based Medicine Center, West China HospitalSichuan UniversityChengduSichuanChina
| | - Xin Sun
- Chinese Evidence‐based Medicine Center, West China HospitalSichuan UniversityChengduSichuanChina
- NMPA Key Laboratory for Real World Data Research and Evaluation in HainanChengduSichuanChina
| |
Collapse
|
28
|
Gao J, Bonzel CL, Hong C, Varghese P, Zakir K, Gronsbell J. Semi-supervised ROC analysis for reliable and streamlined evaluation of phenotyping algorithms. J Am Med Inform Assoc 2024; 31:640-650. [PMID: 38128118 PMCID: PMC10873838 DOI: 10.1093/jamia/ocad226] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 09/22/2023] [Accepted: 11/20/2023] [Indexed: 12/23/2023] Open
Abstract
OBJECTIVE High-throughput phenotyping will accelerate the use of electronic health records (EHRs) for translational research. A critical roadblock is the extensive medical supervision required for phenotyping algorithm (PA) estimation and evaluation. To address this challenge, numerous weakly-supervised learning methods have been proposed. However, there is a paucity of methods for reliably evaluating the predictive performance of PAs when a very small proportion of the data is labeled. To fill this gap, we introduce a semi-supervised approach (ssROC) for estimation of the receiver operating characteristic (ROC) parameters of PAs (eg, sensitivity, specificity). MATERIALS AND METHODS ssROC uses a small labeled dataset to nonparametrically impute missing labels. The imputations are then used for ROC parameter estimation to yield more precise estimates of PA performance relative to classical supervised ROC analysis (supROC) using only labeled data. We evaluated ssROC with synthetic, semi-synthetic, and EHR data from Mass General Brigham (MGB). RESULTS ssROC produced ROC parameter estimates with minimal bias and significantly lower variance than supROC in the simulated and semi-synthetic data. For the 5 PAs from MGB, the estimates from ssROC are 30% to 60% less variable than supROC on average. DISCUSSION ssROC enables precise evaluation of PA performance without demanding large volumes of labeled data. ssROC is also easily implementable in open-source R software. CONCLUSION When used in conjunction with weakly-supervised PAs, ssROC facilitates the reliable and streamlined phenotyping necessary for EHR-based research.
Collapse
Affiliation(s)
- Jianhui Gao
- Department of Statistical Sciences, University of Toronto, Toronto, ON, Canada
| | - Clara-Lea Bonzel
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, United States
| | - Chuan Hong
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, United States
| | - Paul Varghese
- Health Informatics, Verily Life Sciences, Cambridge, MA, United States
| | - Karim Zakir
- Department of Statistical Sciences, University of Toronto, Toronto, ON, Canada
| | - Jessica Gronsbell
- Department of Statistical Sciences, University of Toronto, Toronto, ON, Canada
- Department of Family and Community Medicine, University of Toronto, Toronto, ON, Canada
- Department of Computer Science, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
29
|
Hassan B, Hricz N, Er S, Yoon J, Resnick E, Liang F, Yang R, Manson PN, Grant MP. Development and validation of a risk calculator for postoperative diplopia following orbital fracture repair in adults. Sci Rep 2024; 14:3654. [PMID: 38351033 PMCID: PMC10864303 DOI: 10.1038/s41598-024-54121-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Accepted: 02/08/2024] [Indexed: 02/16/2024] Open
Abstract
Postoperative diplopia is the most common complication following orbital fracture repair (OFR). Existing evidence on its risk factors is based on single-institution studies and small sample sizes. Our study is the first multi-center study to develop and validate a risk calculator for the prediction of postoperative diplopia following OFR. We reviewed trauma patients who underwent OFR at two high-volume trauma centers (2015-2019). Excluded were patients < 18 years old and those with postoperative follow-up < 2 weeks. Our primary outcome was incidence/persistence of postoperative diplopia at ≥ 2 weeks. A risk model for the prediction of postoperative diplopia was derived using a development dataset (70% of population) and validated using a validation dataset (remaining 30%). The C-statistic and Hosmer-Lemeshow tests were used to assess the risk model accuracy. A total of n = 254 adults were analyzed. The factors that predicted postoperative diplopia were: age at injury, preoperative enophthalmos, fracture size/displacement, surgical timing, globe/soft tissue repair, and medial wall involvement. Our predictive model had excellent discrimination (C-statistic = 80.4%), calibration (P = 0.2), and validation (C-statistic = 80%). Our model rules out postoperative diplopia with a 100% sensitivity and negative predictive value (NPV) for a probability < 8.9%. Our predictive model rules out postoperative diplopia with an 87.9% sensitivity and a 95.8% NPV for a probability < 13.4%. We designed the first validated risk calculator that can be used as a powerful screening tool to rule out postoperative diplopia following OFR in adults.
Collapse
Affiliation(s)
- Bashar Hassan
- Division of Plastic and Reconstructive Surgery, R. Adams Cowley Shock Trauma Center, University of Maryland Medical Center, Baltimore, MD, USA
- Department of Plastic and Reconstructive Surgery, Johns Hopkins Hospital, Baltimore, MD, USA
| | - Nicholas Hricz
- University of Maryland School of Medicine, Baltimore, MD, USA
| | - Seray Er
- University of Maryland School of Medicine, Baltimore, MD, USA
| | - Joshua Yoon
- Department of Surgery, George Washington University, Washington, DC, USA
| | - Eric Resnick
- University of Maryland School of Medicine, Baltimore, MD, USA
| | - Fan Liang
- Department of Plastic and Reconstructive Surgery, Johns Hopkins Hospital, Baltimore, MD, USA
| | - Robin Yang
- Department of Plastic and Reconstructive Surgery, Johns Hopkins Hospital, Baltimore, MD, USA
| | - Paul N Manson
- Department of Plastic and Reconstructive Surgery, Johns Hopkins Hospital, Baltimore, MD, USA
| | - Michael P Grant
- Division of Plastic and Reconstructive Surgery, R. Adams Cowley Shock Trauma Center, University of Maryland Medical Center, Baltimore, MD, USA.
- Division of Plastic and Reconstructive Surgery, R. Adams Cowley Shock Trauma Center, University of Maryland Medical Center, 110 S Paca Street, Suite 4-S-124, Baltimore, MD, 21201, USA.
| |
Collapse
|
30
|
Kim HM, Ko T, Kang H, Choi S, Park JH, Chung MK, Kim M, Kim NY, Lee HJ. Improved prediction of clinical pregnancy using artificial intelligence with enhanced inner cell mass and trophectoderm images. Sci Rep 2024; 14:3240. [PMID: 38331914 PMCID: PMC10853203 DOI: 10.1038/s41598-024-52241-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Accepted: 01/16/2024] [Indexed: 02/10/2024] Open
Abstract
This study aimed to assess the performance of an artificial intelligence (AI) model for predicting clinical pregnancy using enhanced inner cell mass (ICM) and trophectoderm (TE) images. In this retrospective study, we included static images of 2555 day-5-blastocysts from seven in vitro fertilization centers in South Korea. The main outcome of the study was the predictive capability of the model to detect clinical pregnancies (gestational sac). Compared with the original embryo images, the use of enhanced ICM and TE images improved the average area under the receiver operating characteristic curve for the AI model from 0.716 to 0.741. Additionally, a gradient-weighted class activation mapping analysis demonstrated that the enhanced image-trained AI model was able to extract features from crucial areas of the embryo in 99% (506/512) of the cases. Particularly, it could extract the ICM and TE. In contrast, the AI model trained on the original images focused on the main areas in only 86% (438/512) of the cases. Our results highlight the potential efficacy of using ICM- and TE-enhanced embryo images when training AI models to predict clinical pregnancy.
Collapse
Affiliation(s)
| | - Taehoon Ko
- Department of Medical Informatics, College of Medicine, The Catholic University of Korea, Seoul, South Korea
- Department of Biomedicine & Health Sciences, College of Medicine, The Catholic University of Korea, Seoul, South Korea
- CMC Institute for Basic Medical Science, The Catholic Medical Center of The Catholic University of Korea, Seoul, South Korea
| | | | | | | | - Mi Kyung Chung
- Seoul Rachel Fertility Center, IVF Clinic, Seoul, South Korea
| | - Miran Kim
- Department of Obstetrics & Gynecology, Ajou University School of Medicine, Suwon, South Korea
| | | | | |
Collapse
|
31
|
Boussen S, Benard-Tertrais M, Ogéa M, Malet A, Simeone P, Antonini F, Bruder N, Velly L. Heart rate complexity helps mortality prediction in the intensive care unit: A pilot study using artificial intelligence. Comput Biol Med 2024; 169:107934. [PMID: 38183707 DOI: 10.1016/j.compbiomed.2024.107934] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2023] [Revised: 12/10/2023] [Accepted: 01/01/2024] [Indexed: 01/08/2024]
Abstract
BACKGROUND In intensive care units (ICUs), accurate mortality prediction is crucial for effective patient management and resource allocation. The Simplified Acute Physiology Score II (SAPS-2), though commonly used, relies heavily on comprehensive clinical data and blood samples. This study sought to develop an artificial intelligence (AI) model utilizing key hemodynamic parameters to predict ICU mortality within the first 24 h and assess its performance relative to SAPS-2. METHODS We conducted an analysis of select hemodynamic parameters and the structure of heart rate curves to identify potential predictors of ICU mortality. A machine-learning model was subsequently trained and validated on distinct patient cohorts. The AI algorithm's performance was then compared to the SAPS-2, focusing on classification accuracy, calibration, and generalizability. MEASUREMENTS AND MAIN RESULTS The study included 1298 ICU admissions from March 27th, 2015, to March 27th, 2017. An additional cohort from 2022 to 2023 comprised 590 patients, resulting in a total dataset of 1888 patients. The observed mortality rate stood at 24.0%. Key determinants of mortality were the Glasgow Coma Scale score, heart rate complexity, patient age, duration of diastolic blood pressure below 50 mmHg, heart rate variability, and specific mean and systolic blood pressure thresholds. The AI model, informed by these determinants, exhibited a performance profile in predicting mortality that was comparable, if not superior, to the SAPS-2. CONCLUSIONS The AI model, which integrates heart rate and blood pressure curve analyses with basic clinical parameters, provides a methodological approach to predict in-hospital mortality in ICU patients. This model offers an alternative to existing tools that depend on extensive clinical data and laboratory inputs. Its potential integration into ICU monitoring systems may facilitate more streamlined mortality prediction processes.
Collapse
Affiliation(s)
- Salah Boussen
- Intensive Care and Anesthesiology Department, La Timone Teaching Hospital, Aix-Marseille Université Assistance Publique Hôpitaux de Marseille, Marseille, France; Laboratoire de Biomécanique Appliquée-Université Gustave-Eiffel, Aix-Marseille Université, UMR T24, 51 boulevard Pierre Dramard, 13015, Marseille, France.
| | - Manuela Benard-Tertrais
- Intensive Care and Anesthesiology Department, La Timone Teaching Hospital, Aix-Marseille Université Assistance Publique Hôpitaux de Marseille, Marseille, France
| | - Mathilde Ogéa
- Intensive Care and Anesthesiology Department, La Timone Teaching Hospital, Aix-Marseille Université Assistance Publique Hôpitaux de Marseille, Marseille, France
| | - Arthur Malet
- Intensive Care and Anesthesiology Department, La Timone Teaching Hospital, Aix-Marseille Université Assistance Publique Hôpitaux de Marseille, Marseille, France
| | - Pierre Simeone
- Intensive Care and Anesthesiology Department, La Timone Teaching Hospital, Aix-Marseille Université Assistance Publique Hôpitaux de Marseille, Marseille, France; Aix Marseille University, CNRS, Inst Neurosci Timone, UMR7289, Marseille, France
| | - François Antonini
- Intensive Care and Anesthesiology Department, Hôpital Nord Teaching Hospital, Aix-Marseille Université Assistance Publique Hôpitaux de Marseille, Marseille, France
| | - Nicolas Bruder
- Intensive Care and Anesthesiology Department, La Timone Teaching Hospital, Aix-Marseille Université Assistance Publique Hôpitaux de Marseille, Marseille, France
| | - Lionel Velly
- Intensive Care and Anesthesiology Department, La Timone Teaching Hospital, Aix-Marseille Université Assistance Publique Hôpitaux de Marseille, Marseille, France; Aix Marseille University, CNRS, Inst Neurosci Timone, UMR7289, Marseille, France
| |
Collapse
|
32
|
Yang J, Huang J, Han D, Ma X. Artificial Intelligence Applications in the Treatment of Colorectal Cancer: A Narrative Review. Clin Med Insights Oncol 2024; 18:11795549231220320. [PMID: 38187459 PMCID: PMC10771756 DOI: 10.1177/11795549231220320] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Accepted: 11/26/2023] [Indexed: 01/09/2024] Open
Abstract
Colorectal cancer is the third most prevalent cancer worldwide, and its treatment has been a demanding clinical problem. Beyond traditional surgical therapy and chemotherapy, newly revealed molecular mechanisms diversify therapeutic approaches for colorectal cancer. However, the selection of personalized treatment among multiple treatment options has become another challenge in the era of precision medicine. Artificial intelligence has recently been increasingly investigated in the treatment of colorectal cancer. This narrative review mainly discusses the applications of artificial intelligence in the treatment of colorectal cancer patients. A comprehensive literature search was conducted in MEDLINE, EMBASE, and Web of Science to identify relevant papers, resulting in 49 articles being included. The results showed that, based on different categories of data, artificial intelligence can predict treatment outcomes and essential guidance information of traditional and novel therapies, thus enabling individualized treatment strategy selection for colorectal cancer patients. Some frequently implemented machine learning algorithms and deep learning frameworks have also been employed for long-term prognosis prediction in patients with colorectal cancer. Overall, artificial intelligence shows encouraging results in treatment strategy selection and prognosis evaluation for colorectal cancer patients.
Collapse
Affiliation(s)
- Jiaqing Yang
- Department of Biotherapy, West China Hospital and State Key Laboratory of Biotherapy, Sichuan University, Chengdu, China
- West China School of Medicine, West China Hospital, Sichuan University, Chengdu, China
| | - Jing Huang
- Department of Ultrasound, West China Hospital, Sichuan University, Chengdu, China
| | - Deqian Han
- Department of Oncology, West China School of Public Health and West China Fourth Hospital, Sichuan University, Chengdu, China
| | - Xuelei Ma
- Department of Biotherapy, West China Hospital and State Key Laboratory of Biotherapy, Sichuan University, Chengdu, China
| |
Collapse
|
33
|
Lei Y, Qiu X, Zhou R. Construction and evaluation of neonatal respiratory failure risk prediction model for neonatal respiratory distress syndrome. BMC Pulm Med 2024; 24:8. [PMID: 38166798 PMCID: PMC10759760 DOI: 10.1186/s12890-023-02819-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2023] [Accepted: 12/18/2023] [Indexed: 01/05/2024] Open
Abstract
BACKGROUND Neonatal respiratory distress syndrome (NRDS) is a common respiratory disease in preterm infants, often accompanied by respiratory failure. The aim of this study was to establish and validate a nomogram model for predicting the probability of respiratory failure in NRDS patients. METHODS Patients diagnosed with NRDS were extracted from the MIMIC-iv database. The patients were randomly assigned to a training and a validation cohort. Univariate and stepwise Cox regression analyses were used to determine the prognostic factors of NRDS. A nomogram containing these factors was established to predict the incidence of respiratory failure in NRDS patients. The area under the receiver operating characteristic curve (AUC), receiver operating characteristic curve (ROC), calibration curves and decision curve analysis were used to determine the effectiveness of this model. RESULTS The study included 2,705 patients with NRDS. Univariate and multivariate stepwise Cox regression analysis showed that the independent risk factors for respiratory failure in NRDS patients were gestational age, pH, partial pressure of oxygen (PO2), partial pressure of carbon dioxide (PCO2), hemoglobin, blood culture, infection, neonatal intracranial hemorrhage, Pulmonary surfactant (PS), parenteral nutrition and respiratory support. Then, the nomogram was constructed and verified. CONCLUSIONS This study identified the independent risk factors of respiratory failure in NRDS patients and used them to construct and evaluate respiratory failure risk prediction model for NRDS. The present findings provide clinicians with the judgment of patients with respiratory failure in NRDS and help clinicians to identify and intervene in the early stage.
Collapse
Affiliation(s)
- Yupeng Lei
- Department of Pediatrics, West China Second University Hospital, Sichuan University, Chengdu, 610041, Sichuan, China
- Key Laboratory of Birth Defects and Related Diseases of Women and Children, Sichuan University, Ministry of Education, Chengdu, 610041, China
| | - Xia Qiu
- Department of Pediatrics, West China Second University Hospital, Sichuan University, Chengdu, 610041, Sichuan, China
- Key Laboratory of Birth Defects and Related Diseases of Women and Children, Sichuan University, Ministry of Education, Chengdu, 610041, China
| | - Ruixi Zhou
- Department of Pediatrics, West China Second University Hospital, Sichuan University, Chengdu, 610041, Sichuan, China.
- Key Laboratory of Birth Defects and Related Diseases of Women and Children, Sichuan University, Ministry of Education, Chengdu, 610041, China.
| |
Collapse
|
34
|
Graf R, Zeldovich M, Friedrich S. Comparing linear discriminant analysis and supervised learning algorithms for binary classification-A method comparison study. Biom J 2024; 66:e2200098. [PMID: 36529690 DOI: 10.1002/bimj.202200098] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2022] [Revised: 11/02/2022] [Accepted: 11/04/2022] [Indexed: 12/23/2022]
Abstract
In psychology, linear discriminant analysis (LDA) is the method of choice for two-group classification tasks based on questionnaire data. In this study, we present a comparison of LDA with several supervised learning algorithms. In particular, we examine to what extent the predictive performance of LDA relies on the multivariate normality assumption. As nonparametric alternatives, the linear support vector machine (SVM), classification and regression tree (CART), random forest (RF), probabilistic neural network (PNN), and the ensemble k conditional nearest neighbor (EkCNN) algorithms are applied. Predictive performance is determined using measures of overall performance, discrimination, and calibration, and is compared in two reference data sets as well as in a simulation study. The reference data are Likert-type data, and comprise 5 and 10 predictor variables, respectively. Simulations are based on the reference data and are done for a balanced and an unbalanced scenario in each case. In order to compare the algorithms' performance, data are simulated from multivariate distributions with differing degrees of nonnormality. Results differ depending on the specific performance measure. The main finding is that LDA is always outperformed by RF in the bimodal data with respect to overall performance. Discriminative ability of the RF algorithm is often higher compared to LDA, but its model calibration is usually worse. Still LDA mostly ranges second in cases it is outperformed by another algorithm, or the differences are only marginal. In consequence, we still recommend LDA for this type of application.
Collapse
Affiliation(s)
- Ricarda Graf
- Department of Mathematics, University of Augsburg, Germany
| | - Marina Zeldovich
- Institute of Medical Psychology and Medical Sociology, University Medical Center Göttingen, Göttingen, Germany
| | - Sarah Friedrich
- Department of Mathematics, University of Augsburg, Germany
- Centre for Advanced Analytics and Predictive Sciences (CAAPS), University of Augsburg, Augsburg, Germany
| |
Collapse
|
35
|
Eickelberg G, Sanchez-Pinto LN, Kline AS, Luo Y. Transportability of bacterial infection prediction models for critically ill patients. J Am Med Inform Assoc 2023; 31:98-108. [PMID: 37647884 PMCID: PMC10746321 DOI: 10.1093/jamia/ocad174] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2023] [Revised: 07/20/2023] [Accepted: 08/16/2023] [Indexed: 09/01/2023] Open
Abstract
OBJECTIVE Bacterial infections (BIs) are common, costly, and potentially life-threatening in critically ill patients. Patients with suspected BIs may require empiric multidrug antibiotic regimens and therefore potentially be exposed to prolonged and unnecessary antibiotics. We previously developed a BI risk model to augment practices and help shorten the duration of unnecessary antibiotics to improve patient outcomes. Here, we have performed a transportability assessment of this BI risk model in 2 tertiary intensive care unit (ICU) settings and a community ICU setting. We additionally explored how simple multisite learning techniques impacted model transportability. METHODS Patients suspected of having a community-acquired BI were identified in 3 datasets: Medical Information Mart for Intensive Care III (MIMIC), Northwestern Medicine Tertiary (NM-T) ICUs, and NM "community-based" ICUs. ICU encounters from MIMIC and NM-T datasets were split into 70/30 train and test sets. Models developed on training data were evaluated against the NM-T and MIMIC test sets, as well as NM community validation data. RESULTS During internal validations, models achieved AUROCs of 0.78 (MIMIC) and 0.81 (NM-T) and were well calibrated. In the external community ICU validation, the NM-T model had robust transportability (AUROC 0.81) while the MIMIC model transported less favorably (AUROC 0.74), likely due to case-mix differences. Multisite learning provided no significant discrimination benefit in internal validation studies but offered more stability during transport across all evaluation datasets. DISCUSSION These results suggest that our BI risk models maintain predictive utility when transported to external cohorts. CONCLUSION Our findings highlight the importance of performing external model validation on myriad clinically relevant populations prior to implementation.
Collapse
Affiliation(s)
- Garrett Eickelberg
- Department of Preventive Medicine (Health & Biomedical Informatics), Feinberg School of Medicine, Chicago, IL 60611, United States
| | - Lazaro Nelson Sanchez-Pinto
- Department of Preventive Medicine (Health & Biomedical Informatics), Feinberg School of Medicine, Chicago, IL 60611, United States
- Departments of Pediatrics (Critical Care), Chicago, IL 60611, United States
| | - Adrienne Sarah Kline
- Department of Preventive Medicine (Health & Biomedical Informatics), Feinberg School of Medicine, Chicago, IL 60611, United States
| | - Yuan Luo
- Department of Preventive Medicine (Health & Biomedical Informatics), Feinberg School of Medicine, Chicago, IL 60611, United States
| |
Collapse
|
36
|
Ojeda FM, Jansen ML, Thiéry A, Blankenberg S, Weimar C, Schmid M, Ziegler A. Calibrating machine learning approaches for probability estimation: A comprehensive comparison. Stat Med 2023; 42:5451-5478. [PMID: 37849356 DOI: 10.1002/sim.9921] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2021] [Revised: 08/30/2023] [Accepted: 09/18/2023] [Indexed: 10/19/2023]
Abstract
Statistical prediction models have gained popularity in applied research. One challenge is the transfer of the prediction model to a different population which may be structurally different from the model for which it has been developed. An adaptation to the new population can be achieved by calibrating the model to the characteristics of the target population, for which numerous calibration techniques exist. In view of this diversity, we performed a systematic evaluation of various popular calibration approaches used by the statistical and the machine learning communities for estimating two-class probabilities. In this work, we first provide a review of the literature and, second, present the results of a comprehensive simulation study. The calibration approaches are compared with respect to their empirical properties and relationships, their ability to generalize precise probability estimates to external populations and their availability in terms of easy-to-use software implementations. Third, we provide code from real data analysis allowing its application by researchers. Logistic calibration and beta calibration, which estimate an intercept plus one and two slope parameters, respectively, consistently showed the best results in the simulation studies. Calibration on logit transformed probability estimates generally outperformed calibration methods on nontransformed estimates. In case of structural differences between training and validation data, re-estimation of the entire prediction model should be outweighted against sample size of the validation data. We recommend regression-based calibration approaches using transformed probability estimates, where at least one slope is estimated in addition to an intercept for updating probability estimates in validation studies.
Collapse
Affiliation(s)
- Francisco M Ojeda
- Department of Cardiology, University Heart and Vascular Center Hamburg, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
- Centre for Population Health Innovation (POINT), University Heart and Vascular Center Hamburg, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Max L Jansen
- Cardio-CARE, Medizincampus Davos, Davos, Switzerland
| | | | - Stefan Blankenberg
- Department of Cardiology, University Heart and Vascular Center Hamburg, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
- Centre for Population Health Innovation (POINT), University Heart and Vascular Center Hamburg, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
- German Center for Cardiovascular Research (DZHK), Partner Site Hamburg/Kiel/Lübeck, Hamburg, Germany
| | - Christian Weimar
- BDH-Klinik Elzach, Baden-Wuerttemberg, Germany
- Institute for Medical Informatics, Biometry and Epidemiology, University of Duisburg-Essen, Essen, North Rhine-Westphalia, Germany
| | - Matthias Schmid
- Institute of Medical Biometry, Informatics and Epidemiology, Faculty of Medicine, University of Bonn, Bonn, North Rhine-Westphalia, Germany
| | - Andreas Ziegler
- Department of Cardiology, University Heart and Vascular Center Hamburg, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
- Centre for Population Health Innovation (POINT), University Heart and Vascular Center Hamburg, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
- Cardio-CARE, Medizincampus Davos, Davos, Switzerland
- School of Mathematics, Statistics and Computer Science, University of KwaZulu-Natal, Pietermaritzburg, South Africa
- Swiss Institute of Bioinformatics, Lausanne, Waadt, Switzerland
| |
Collapse
|
37
|
Jayaram N, Allen P, Hall M, Karamlou T, Woo J, Crook S, Anderson BR. Adjusting for Congenital Heart Surgery Risk Using Administrative Data. J Am Coll Cardiol 2023; 82:2212-2221. [PMID: 38030351 DOI: 10.1016/j.jacc.2023.09.826] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Revised: 09/11/2023] [Accepted: 09/26/2023] [Indexed: 12/01/2023]
Abstract
BACKGROUND Congenital heart surgery (CHS) encompasses a heterogeneous population of patients and surgeries. Risk standardization models that adjust for patient and procedural characteristics can allow for collective study of these disparate patients and procedures. OBJECTIVES We sought to develop a risk-adjustment model for CHS using the newly developed Risk Stratification for Congenital Heart Surgery for ICD-10 Administrative Data (RACHS-2) methodology. METHODS Within the Kids' Inpatient Database 2019, we identified all CHSs that could be assigned a RACHS-2 score. Hierarchical logistic regression (clustered on hospital) was used to identify patient and procedural characteristics associated with in-hospital mortality. Model validation was performed using data from 24 State Inpatient Databases during 2017. RESULTS Of 5,902,538 total weighted hospital discharges in the Kids' Inpatient Database 2019, 22,310 pediatric cardiac surgeries were identified and assigned a RACHS-2 score. In-hospital mortality occurred in 543 (2.4%) of cases. Using only RACHS-2, the mortality mode had a C-statistic of 0.81 that improved to 0.83 with the addition of age. A final multivariable model inclusive of RACHS-2, age, payer, and presence of a complex chronic condition outside of congenital heart disease further improved model discrimination to 0.87 (P < 0.001). Discrimination in the validation cohort was also very good with a C-statistic of 0.83. CONCLUSIONS We created and validated a risk-adjustment model for CHS that accounts for patient and procedural characteristics associated with in-hospital mortality available in administrative data, including the newly developed RACHS-2. Our risk model will be critical for use in health services research and quality improvement initiatives.
Collapse
Affiliation(s)
| | - Philip Allen
- Brigham and Women's Hospital, Boston, Massachusetts, USA
| | - Matthew Hall
- Children's Hospital Association, Lenexa, Kansas, USA
| | | | - Joyce Woo
- Lurie Children's Hospital, Chicago, Illinois, USA
| | - Sarah Crook
- NewYork-Presbyterian/Columbia University Irving Medical Center, New York, New York, USA
| | - Brett R Anderson
- NewYork-Presbyterian/Columbia University Irving Medical Center, New York, New York, USA
| |
Collapse
|
38
|
Husain SA. A Responsibility to Perpetually Seek Improved Risk Stratification Models: Achieving Data Nirvana. J Am Coll Cardiol 2023; 82:2222-2224. [PMID: 38030352 DOI: 10.1016/j.jacc.2023.10.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Accepted: 10/10/2023] [Indexed: 12/01/2023]
Affiliation(s)
- S Adil Husain
- Pediatric Cardiothoracic Surgery, University of Utah Health, Salt Lake City, Utah, USA; Heart Center, Primary Children's Hospital, Salt Lake City, Utah, USA.
| |
Collapse
|
39
|
Yuan N, Duffy G, Dhruva SS, Oesterle A, Pellegrini CN, Theurer J, Vali M, Heidenreich PA, Keyhani S, Ouyang D. Deep Learning of Electrocardiograms in Sinus Rhythm From US Veterans to Predict Atrial Fibrillation. JAMA Cardiol 2023; 8:1131-1139. [PMID: 37851434 PMCID: PMC10585587 DOI: 10.1001/jamacardio.2023.3701] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/22/2023] [Accepted: 07/31/2023] [Indexed: 10/19/2023]
Abstract
Importance Early detection of atrial fibrillation (AF) may help prevent adverse cardiovascular events such as stroke. Deep learning applied to electrocardiograms (ECGs) has been successfully used for early identification of several cardiovascular diseases. Objective To determine whether deep learning models applied to outpatient ECGs in sinus rhythm can predict AF in a large and diverse patient population. Design, Setting, and Participants This prognostic study was performed on ECGs acquired from January 1, 1987, to December 31, 2022, at 6 US Veterans Affairs (VA) hospital networks and 1 large non-VA academic medical center. Participants included all outpatients with 12-lead ECGs in sinus rhythm. Main Outcomes and Measures A convolutional neural network using 12-lead ECGs from 2 US VA hospital networks was trained to predict the presence of AF within 31 days of sinus rhythm ECGs. The model was tested on ECGs held out from training at the 2 VA networks as well as 4 additional VA networks and 1 large non-VA academic medical center. Results A total of 907 858 ECGs from patients across 6 VA sites were included in the analysis. These patients had a mean (SD) age of 62.4 (13.5) years, 6.4% were female, and 93.6% were male, with a mean (SD) CHA2DS2-VASc (congestive heart failure, hypertension, age, diabetes mellitus, prior stroke or transient ischemic attack or thromboembolism, vascular disease, age, sex category) score of 1.9 (1.6). A total of 0.2% were American Indian or Alaska Native, 2.7% were Asian, 10.7% were Black, 4.6% were Latinx, 0.7% were Native Hawaiian or Other Pacific Islander, 62.4% were White, 0.4% were of other race or ethnicity (which is not broken down into subcategories in the VA data set), and 18.4% were of unknown race or ethnicity. At the non-VA academic medical center (72 483 ECGs), the mean (SD) age was 59.5 (15.4) years and 52.5% were female, with a mean (SD) CHA2DS2-VASc score of 1.6 (1.4). A total of 0.1% were American Indian or Alaska Native, 7.9% were Asian, 9.4% were Black, 2.9% were Latinx, 0.03% were Native Hawaiian or Other Pacific Islander, 74.8% were White, 0.1% were of other race or ethnicity, and 4.7% were of unknown race or ethnicity. A deep learning model predicted the presence of AF within 31 days of a sinus rhythm ECG on held-out test ECGs at VA sites with an area under the receiver operating characteristic curve (AUROC) of 0.86 (95% CI, 0.85-0.86), accuracy of 0.78 (95% CI, 0.77-0.78), and F1 score of 0.30 (95% CI, 0.30-0.31). At the non-VA site, AUROC was 0.93 (95% CI, 0.93-0.94); accuracy, 0.87 (95% CI, 0.86-0.88); and F1 score, 0.46 (95% CI, 0.44-0.48). The model was well calibrated, with a Brier score of 0.02 across all sites. Among individuals deemed high risk by deep learning, the number needed to screen to detect a positive case of AF was 2.47 individuals for a testing sensitivity of 25% and 11.48 for 75%. Model performance was similar in patients who were Black, female, or younger than 65 years or who had CHA2DS2-VASc scores of 2 or greater. Conclusions and Relevance Deep learning of outpatient sinus rhythm ECGs predicted AF within 31 days in populations with diverse demographics and comorbidities. Similar models could be used in future AF screening efforts to reduce adverse complications associated with this disease.
Collapse
Affiliation(s)
- Neal Yuan
- Department of Medicine, University of California, San Francisco
- Division of Cardiology, San Francisco Veterans Affairs Medical Center, San Francisco, California
| | - Grant Duffy
- Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, California
- Division of Artificial Intelligence in Medicine, Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, California
| | - Sanket S. Dhruva
- Department of Medicine, University of California, San Francisco
- Division of Cardiology, San Francisco Veterans Affairs Medical Center, San Francisco, California
| | - Adam Oesterle
- Department of Medicine, University of California, San Francisco
- Division of Cardiology, San Francisco Veterans Affairs Medical Center, San Francisco, California
| | - Cara N. Pellegrini
- Department of Medicine, University of California, San Francisco
- Division of Cardiology, San Francisco Veterans Affairs Medical Center, San Francisco, California
| | - John Theurer
- Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, California
- Division of Artificial Intelligence in Medicine, Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, California
| | - Marzieh Vali
- Department of Medicine, University of California, San Francisco
- Division of General Internal Medicine, San Francisco Veterans Affairs Medical Center, San Francisco, California
| | - Paul A. Heidenreich
- Division of Cardiology, Palo Alto Veterans Affairs Medical Center, Palo Alto, California
- Division of Cardiovascular Medicine, Department of Medicine, Stanford University, Palo Alto, California
| | - Salomeh Keyhani
- Department of Medicine, University of California, San Francisco
- Division of General Internal Medicine, San Francisco Veterans Affairs Medical Center, San Francisco, California
| | - David Ouyang
- Division of Cardiology, San Francisco Veterans Affairs Medical Center, San Francisco, California
- Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, California
| |
Collapse
|
40
|
Qiu J, Chang Z, Wang K, Chen K, Wang Q, Zhang J, Li J, Yang C, Zhao Y, Zhang Y. The predictive accuracy of coronary heart disease risk prediction models in rural Northwestern China. Prev Med Rep 2023; 36:102503. [PMID: 38116288 PMCID: PMC10728432 DOI: 10.1016/j.pmedr.2023.102503] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Revised: 11/06/2023] [Accepted: 11/07/2023] [Indexed: 12/21/2023] Open
Abstract
Cardiovascular risk models developed may have limitations when applied to rural Chinese. This study validated and compared the Framingham Risk Score (FRS) and Prediction for Atherosclerotic Cardiovascular Disease Risk in China (PAR) models in predicting 10-year risk of coronary heart disease (CHD) in a rural cohort in Ningxia, China from 2008 to 2019. The FRS and PAR models were validated by estimating predicted events, C index, calibration χ2 and plots. 1381 adults without CHD at baseline were followed up for 9.75 years on average. 168 CHD cases were observed. The FRS and PAR underestimated CHD events by 22 % and 46 % for the total population, while overestimated for males by 152 % and 78 %, respectively. The C index was slightly higher for PAR than FRS. Both models showed weak calibration with chi-square values above 20 (p < 0.001). Bland-Altman plots indicated FRS predicted higher CHD risk than PAR, lacking consistency. Overall, FRS and PAR demonstrated limited performance in predicting 10-year CHD risk in this rural population. PAR had slightly better discrimination than FRS, but require further improvement in calibration and individual risk estimation to suit the rural population in Northwest China.
Collapse
Affiliation(s)
- Jiangwei Qiu
- School of Public, Ningxia Medical University, Yinchuan, China
- NHC Key Laboratory of Metabolic Cardiovascular Diseases Research, Ningxia Medical University, Yinchuan, China
| | - Zhenqi Chang
- School of Public, Ningxia Medical University, Yinchuan, China
| | - Kai Wang
- School of Public, Ningxia Medical University, Yinchuan, China
- The Key Laboratory of Environmental Factors and Chronic Disease Control, Ningxia Medical University, Yinchuan, China
| | - Kexin Chen
- School of Public, Ningxia Medical University, Yinchuan, China
| | - Qingan Wang
- School of Public, Ningxia Medical University, Yinchuan, China
- The Key Laboratory of Environmental Factors and Chronic Disease Control, Ningxia Medical University, Yinchuan, China
| | - Jiaxing Zhang
- School of Public, Ningxia Medical University, Yinchuan, China
- The Key Laboratory of Environmental Factors and Chronic Disease Control, Ningxia Medical University, Yinchuan, China
| | - Juan Li
- School of Public, Ningxia Medical University, Yinchuan, China
| | - Chan Yang
- School of Public, Ningxia Medical University, Yinchuan, China
- Department of Community Nursing, School of Nursing, Ningxia Medical University, Yinchuan, China
| | - Yi Zhao
- School of Public, Ningxia Medical University, Yinchuan, China
- NHC Key Laboratory of Metabolic Cardiovascular Diseases Research, Ningxia Medical University, Yinchuan, China
- The Key Laboratory of Environmental Factors and Chronic Disease Control, Ningxia Medical University, Yinchuan, China
| | - Yuhong Zhang
- School of Public, Ningxia Medical University, Yinchuan, China
- NHC Key Laboratory of Metabolic Cardiovascular Diseases Research, Ningxia Medical University, Yinchuan, China
- The Key Laboratory of Environmental Factors and Chronic Disease Control, Ningxia Medical University, Yinchuan, China
| |
Collapse
|
41
|
Liu T, Zhao Z, Wu C, Lu C, Liu M, An X, Sha Z, Wang X, Luo Z, Chen L, Liu C, Cao P, Zhang D, Jiang R. Impact of COVID-19 infection experience on mental health status of intensive care unit patients' family members: a real-world study. QJM 2023; 116:903-910. [PMID: 37498557 DOI: 10.1093/qjmed/hcad184] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/10/2023] [Revised: 07/05/2023] [Indexed: 07/28/2023] Open
Abstract
PURPOSE Family members of patients hospitalized in intensive care unit (ICU) are susceptible to adverse psychological outcomes. However, there is a paucity of studies specifically examining the mental health symptoms in ICU patients' family members with a prior history of coronavirus disease 2019 (COVID-19) infection. AIM This study aimed to investigate mental health status and its influencing factors of ICU patients' family members with previous COVID-19 infection experience in China. DESIGN Nationwide, cross-sectional cohort of consecutive participants of family members of ICU patients from 10 provinces randomly selected in mainland China conducted between October 2022 and May 2023. METHODS The basic information scale, Self-rating depression scale, Self-rating Anxiety Scale, Impact of Event Scale-Revised, Pittsburgh sleep quality index, Perceived Stress Scale, Connor-Davidson resilience scale, Simplified Coping Style Questionnaire were employed to explore mental health status among participants. RESULTS A total of 463 participants, comprising 156 individuals in Covid-19 family member cohort (infection group) and 307 individuals in control family member cohort (control group), met inclusion criteria. The infection group exhibited significantly higher incidence of composite mental health symptoms compared to control group (P = 0.017). Multivariable logistic regression analysis revealed that being female, engaging in physical/mental labor, residing in rural areas, and having children were identified as risk factors for the development of depression, anxiety, and post-traumatic stress disorder symptoms, while medical history of surgery was protective factor. A predictive model demonstrated a favorable discriminative ability and excellent calibration. CONCLUSION COVID-19 infection experience regarded as new traumatic stressors worsen mental health status of ICU patients' family members.
Collapse
Affiliation(s)
- T Liu
- Department of Neurosurgery, Tianjin Medical University General Hospital, Tianjin, China
- Key Laboratory of Post Neuro-Injury Neuro-Repair and Regeneration in Central Nervous System, Tianjin Neurological Institute, Tianjin Medical University General Hospital, Ministry of Education, Tianjin, China
| | - Z Zhao
- Department of Neurosurgery, Tianjin Medical University General Hospital, Tianjin, China
- Key Laboratory of Post Neuro-Injury Neuro-Repair and Regeneration in Central Nervous System, Tianjin Neurological Institute, Tianjin Medical University General Hospital, Ministry of Education, Tianjin, China
| | - C Wu
- Department of Neurosurgery, Tianjin Medical University General Hospital, Tianjin, China
- Key Laboratory of Post Neuro-Injury Neuro-Repair and Regeneration in Central Nervous System, Tianjin Neurological Institute, Tianjin Medical University General Hospital, Ministry of Education, Tianjin, China
| | - C Lu
- Department of Psychiatry, Tianjin Anding Hospital, Mental Health Center of Tianjin Medical University, Tianjin, China
| | - M Liu
- Department of Neurosurgery, Tianjin Medical University General Hospital, Tianjin, China
- Key Laboratory of Post Neuro-Injury Neuro-Repair and Regeneration in Central Nervous System, Tianjin Neurological Institute, Tianjin Medical University General Hospital, Ministry of Education, Tianjin, China
| | - X An
- Department of Intensive Care Unit, Beijing Tiantan Hospital, Beijing, China
| | - Z Sha
- Department of Neurosurgery, Tianjin Medical University General Hospital, Tianjin, China
- Key Laboratory of Post Neuro-Injury Neuro-Repair and Regeneration in Central Nervous System, Tianjin Neurological Institute, Tianjin Medical University General Hospital, Ministry of Education, Tianjin, China
| | - X Wang
- Department of Neurology, The First Affiliated Hospital of China Medical University, Shenyang, China
| | - Z Luo
- Department of Neurosurgery, Shandong Provincial Third Hospital, Shandong, China
| | - L Chen
- Department of Intensive Care Unit, Shaoxing People's Hospital, Shaoxing, Zhejiang, China
| | - C Liu
- Department of Radiology, The First Affiliated Hospital of Anhui Medical University, Anhui, China
| | - P Cao
- Department of Intensive Care Unit, The First Affiliated Hospital of Bengbu Medical College, Anhui, China
| | - D Zhang
- Tianjin Neurological Institute, Key Laboratory of Post Neuro-Injury Neuro-Repair and Regeneration in Central Nervous System, Ministry of Education and Tianjin City, Tianjin, 300052, China
| | - R Jiang
- Department of Neurosurgery, Tianjin Medical University General Hospital, Tianjin, China
- Key Laboratory of Post Neuro-Injury Neuro-Repair and Regeneration in Central Nervous System, Tianjin Neurological Institute, Tianjin Medical University General Hospital, Ministry of Education, Tianjin, China
| |
Collapse
|
42
|
Yagin FH, Alkhateeb A, Raza A, Samee NA, Mahmoud NF, Colak C, Yagin B. An Explainable Artificial Intelligence Model Proposed for the Prediction of Myalgic Encephalomyelitis/Chronic Fatigue Syndrome and the Identification of Distinctive Metabolites. Diagnostics (Basel) 2023; 13:3495. [PMID: 38066735 PMCID: PMC10706650 DOI: 10.3390/diagnostics13233495] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2023] [Revised: 11/03/2023] [Accepted: 11/17/2023] [Indexed: 06/28/2024] Open
Abstract
BACKGROUND Myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) is a complex and debilitating illness with a significant global prevalence, affecting over 65 million individuals. It affects various systems, including the immune, neurological, gastrointestinal, and circulatory systems. Studies have shown abnormalities in immune cell types, increased inflammatory cytokines, and brain abnormalities. Further research is needed to identify consistent biomarkers and develop targeted therapies. This study uses explainable artificial intelligence and machine learning techniques to identify discriminative metabolites for ME/CFS. MATERIAL AND METHODS The model investigates a metabolomics dataset of CFS patients and healthy controls, including 26 healthy controls and 26 ME/CFS patients aged 22-72. The dataset encapsulated 768 metabolites into nine metabolic super-pathways: amino acids, carbohydrates, cofactors, vitamins, energy, lipids, nucleotides, peptides, and xenobiotics. Random forest methods together with other classifiers were applied to the data to classify individuals as ME/CFS patients and healthy individuals. The classification learning algorithms' performance in the validation step was evaluated using a variety of methods, including the traditional hold-out validation method, as well as the more modern cross-validation and bootstrap methods. Explainable artificial intelligence approaches were applied to clinically explain the optimum model's prediction decisions. RESULTS The metabolomics of C-glycosyltryptophan, oleoylcholine, cortisone, and 3-hydroxydecanoate were determined to be crucial for ME/CFS diagnosis. The random forest model outperformed the other classifiers in ME/CFS prediction using the 1000-iteration bootstrapping method, achieving 98% accuracy, precision, recall, F1 score, 0.01 Brier score, and 99% AUC. According to the obtained results, the bootstrap validation approach demonstrated the highest classification outcomes. CONCLUSION The proposed model accurately classifies ME/CFS patients based on the selected biomarker candidate metabolites. It offers a clear interpretation of risk estimation for ME/CFS, aiding physicians in comprehending the significance of key metabolomic features within the model.
Collapse
Affiliation(s)
- Fatma Hilal Yagin
- Department of Biostatistics and Medical Informatics, Faculty of Medicine, Inonu University, Malatya 44280, Türkiye;
| | | | - Ali Raza
- Institute of Computer Science, Khwaja Fareed University of Engineering and Information Technology, Rahim Yar Khan 64200, Pakistan;
| | - Nagwan Abdel Samee
- Department of Information Technology, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia;
| | - Noha F. Mahmoud
- Rehabilitation Sciences Department, Health and Rehabilitation Sciences College, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia;
| | - Cemil Colak
- Department of Biostatistics and Medical Informatics, Faculty of Medicine, Inonu University, Malatya 44280, Türkiye;
| | - Burak Yagin
- Department of Biostatistics and Medical Informatics, Faculty of Medicine, Inonu University, Malatya 44280, Türkiye;
| |
Collapse
|
43
|
Schuch HS, Furtado M, Silva GFDS, Kawachi I, Chiavegatto Filho ADP, Elani HW. Fairness of Machine Learning Algorithms for Predicting Foregone Preventive Dental Care for Adults. JAMA Netw Open 2023; 6:e2341625. [PMID: 37921762 PMCID: PMC10625037 DOI: 10.1001/jamanetworkopen.2023.41625] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/02/2023] [Accepted: 09/15/2023] [Indexed: 11/04/2023] Open
Abstract
Importance Access to routine dental care prevents advanced dental disease and improves oral and overall health. Identifying individuals at risk of foregoing preventive dental care can direct prevention efforts toward high-risk populations. Objective To predict foregone preventive dental care among adults overall and in sociodemographic subgroups and to assess the algorithmic fairness. Design, Setting, and Participants This prognostic study was a secondary analyses of longitudinal data from the US Medical Expenditure Panel Survey (MEPS) from 2016 to 2019, each with 2 years of follow-up. Participants included adults aged 18 years and older. Data analysis was performed from December 2022 to June 2023. Exposure A total of 50 predictors, including demographic and socioeconomic characteristics, health conditions, behaviors, and health services use, were assessed. Main Outcomes and Measures The outcome of interest was foregoing preventive dental care, defined as either cleaning, general examination, or an appointment with the dental hygienist, in the past year. Results Among 32 234 participants, the mean (SD) age was 48.5 (18.2) years and 17 386 participants (53.9%) were female; 1935 participants (6.0%) were Asian, 5138 participants (15.9%) were Black, 7681 participants (23.8%) were Hispanic, 16 503 participants (51.2%) were White, and 977 participants (3.0%) identified as other (eg, American Indian and Alaska Native) or multiple racial or ethnic groups. There were 21 083 (65.4%) individuals who missed preventive dental care in the past year. The algorithms demonstrated high performance, achieving an area under the receiver operating characteristic curve (AUC) of 0.84 (95% CI, 0.84-0.85) in the overall population. While the full sample model performed similarly when applied to White individuals and older adults (AUC, 0.88; 95% CI, 0.87-0.90), there was a loss of performance for other subgroups. Removing the subgroup-sensitive predictors (ie, race and ethnicity, age, and income) did not impact model performance. Models stratified by race and ethnicity performed similarly or worse than the full model for all groups, with the lowest performance for individuals who identified as other or multiple racial groups (AUC, 0.76; 95% CI, 0.70-0.81). Previous pattern of dental visits, health care utilization, dental benefits, and sociodemographic characteristics were the highest contributing predictors to the models' performance. Conclusions and Relevance Findings of this prognostic study using cohort data suggest that tree-based ensemble machine learning models could accurately predict adults at risk of foregoing preventive dental care and demonstrated bias against underrepresented sociodemographic groups. These results highlight the importance of evaluating model fairness during development and testing to avoid exacerbating existing biases.
Collapse
Affiliation(s)
| | - Mariane Furtado
- Harvard School of Dental Medicine, Harvard University, Boston, Massachusetts
| | | | - Ichiro Kawachi
- Harvard T. H. Chan School of Public Health, Harvard University, Boston, Massachusetts
| | | | - Hawazin W. Elani
- Harvard School of Dental Medicine, Harvard University, Boston, Massachusetts
- Harvard T. H. Chan School of Public Health, Harvard University, Boston, Massachusetts
| |
Collapse
|
44
|
Zhong D, Wang Y, Lin L, Cheng S, Zhao GS, Wang LY, Liu Y, Ke ZY. Development and Validation of a Nomogram to Predict the Risk of Recurrent Lower Extremity Radiating Pain Within 1 Week Following Full-Endoscopic Lumbar Discectomy. World Neurosurg 2023; 179:e348-e358. [PMID: 37634669 DOI: 10.1016/j.wneu.2023.08.090] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2023] [Accepted: 08/22/2023] [Indexed: 08/29/2023]
Abstract
BACKGROUND Accurately predicting the risk of lower extremity (LE) radiating pain after surgery is an important endeavor for spinal surgeons. Our study aimed to identify risk factors for LE radiating pain after decompression with full-endoscopic lumbar discectomy (FELD) and develop a nomogram. METHODS We retrospectively reviewed the medical data of patients with lumbar disc herniation who underwent FELD. Two hundred thirty-five patients diagnosed at our hospital from January 2015 to December 2020 were used for model development. The independent risk factors for LE radiating pain after surgery were determined by least absolute shrinkage and selection operator logistic regression and multivariate logistic regression analysis. A nomogram was developed to predict the risk of LE radiating pain based on independent risk factors. Receiver operating characteristic curve, calibration curve, and decision curve analyses were used to evaluate the predictive performance. The nomogram was further verified by an independent cohort. RESULTS Three hundred seventy-five patients were enrolled in this study, with 102 patients in the training cohort reporting LE radiating pain after FELD, while 133 patients did not. In the validation cohort, 57 patients reported LE radiating pain after FELD, while 83 patients did not. The model was established by multivariate logistic regression analysis. The risk factors included a higher Michigan State University classification of herniated discs, increased disease course, increased time of surgery, reduced lateral recess width, and an interlaminar surgical approach, compared to transforaminal approach. The C-indices and the area under the receiver operating characteristic curve of the predictive model demonstrated good discrimination. Good predictive performance and accuracy were also observed in the validation cohort. CONCLUSIONS A novel nomogram for predicting recurrent LE radiating pain within 1 week after FELD was established and validated. More aggressive pain management strategies should be considered for patients at high risk of LE radiating pain after surgery, as predicted by this model.
Collapse
Affiliation(s)
- Dian Zhong
- Department of Spine Surgery, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, China
| | - Yang Wang
- Department of Spine Surgery, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, China.
| | - Lu Lin
- Department of Spine Surgery, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, China
| | - Si Cheng
- Department of Spine Surgery, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, China
| | - Guo Sheng Zhao
- Department of Spine Surgery, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, China
| | - Li Yuan Wang
- Department of Spine Surgery, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, China
| | - Yang Liu
- Department of Spine Surgery, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, China
| | - Zhen Yong Ke
- Department of Spine Surgery, The Second Affiliated Hospital of Chongqing Medical University, Chongqing, China
| |
Collapse
|
45
|
Mitsuke A, Ohbo T, Arima J, Osako Y, Sakaguchi T, Matsushita R, Yoshino H, Tatarano S, Yamada Y, Sasaki H, Tanabe T, Fukuzawa N, Tanaka H, Nishio Y, Hideki E, Harada H. Low dose tacrolimus exposure and early steroid withdrawal with strict body weight control can improve post kidney transplant glucose tolerance in Japanese patients. PLoS One 2023; 18:e0287059. [PMID: 37819994 PMCID: PMC10566682 DOI: 10.1371/journal.pone.0287059] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2022] [Accepted: 05/26/2023] [Indexed: 10/13/2023] Open
Abstract
The development of diabetes mellitus (DM) after living donor kidney transplantation (KT) is a risk factor for worsening transplant kidney function, cardiac disease, and cerebrovascular disease, which may affect prognosis after KT. At our institution, all patients' glucose tolerance is evaluated perioperatively by oral glucose tolerance tests (OGTTs) at pre-KT, and 3, 6, and 12 month (mo.) after KT. We analyzed the insulinogenic index (ISI) and homeostasis model assessment beta cell (HOMA-β) based on the immunoreactive insulin (IRI) levels to determine how glucose tolerance changed after KT in 214 patients who had not been diagnosed with DM before KT. In addition, we analyzed the body mass index (BMI) which may also influence glucose tolerance after KT. The concentration of tacrolimus (TAC) in blood was also measured as the area under the curve (AUC) to examine its effects at each sampling point. The preoperative-OGTTs showed that DM was newly diagnosed in 22 of 214 patients (10.3%) who had not been given a diagnosis of DM by the pre-KT fasting blood sugar (FBS) tests. The glucose tolerance was improved in 15 of 22 DM patients at 12 mo. after KT. ISI and IRI deteriorated only at 3 mo. after KT but improved over time. There was a trend of an inverse correlation between HOMA-β and TAC-AUC. We also found inverse correlations between IRI and an increase in BMI from 3 to 12 mo. after KT. Early corticosteroid withdrawal or the steroid minimization protocol with tacrolimus to maintain a low level of diabetogenic tacrolimus and BMI decrease after KT used by our hospital individualizes lifestyle interventions for each patient might contribute to an improvement in post-KT glucose tolerance.
Collapse
Affiliation(s)
- Akihiko Mitsuke
- Department of Urology, Graduate of School of Medical and Dental Sciences, Kagoshima University, Kagoshima, Japan
- Department of Kidney Transplant Surgery, Sapporo City General Hospital, Hokkaido, Japan
| | - Takahiko Ohbo
- Department of Diabetes and Endocrine Medicine, Graduate of School of Medical and Dental Sciences, Kagoshima University, Kagoshima, Japan
| | - Junya Arima
- Department of Urology, Graduate of School of Medical and Dental Sciences, Kagoshima University, Kagoshima, Japan
| | - Yoichi Osako
- Department of Urology, Graduate of School of Medical and Dental Sciences, Kagoshima University, Kagoshima, Japan
| | - Takashi Sakaguchi
- Department of Urology, Graduate of School of Medical and Dental Sciences, Kagoshima University, Kagoshima, Japan
| | - Ryosuke Matsushita
- Department of Urology, Graduate of School of Medical and Dental Sciences, Kagoshima University, Kagoshima, Japan
| | - Hirofumi Yoshino
- Department of Urology, Graduate of School of Medical and Dental Sciences, Kagoshima University, Kagoshima, Japan
| | - Shuichi Tatarano
- Department of Urology, Graduate of School of Medical and Dental Sciences, Kagoshima University, Kagoshima, Japan
| | - Yasutoshi Yamada
- Department of Urology, Graduate of School of Medical and Dental Sciences, Kagoshima University, Kagoshima, Japan
| | - Hajime Sasaki
- Department of Kidney Transplant Surgery, Sapporo City General Hospital, Hokkaido, Japan
| | - Tatsu Tanabe
- Department of Kidney Transplant Surgery, Sapporo City General Hospital, Hokkaido, Japan
| | - Nobuyuki Fukuzawa
- Department of Kidney Transplant Surgery, Sapporo City General Hospital, Hokkaido, Japan
| | - Hiroshi Tanaka
- Department of Kidney Transplant Surgery, Sapporo City General Hospital, Hokkaido, Japan
| | - Yoshihiko Nishio
- Department of Diabetes and Endocrine Medicine, Graduate of School of Medical and Dental Sciences, Kagoshima University, Kagoshima, Japan
| | - Enokida Hideki
- Department of Urology, Graduate of School of Medical and Dental Sciences, Kagoshima University, Kagoshima, Japan
| | - Hiroshi Harada
- Department of Kidney Transplant Surgery, Sapporo City General Hospital, Hokkaido, Japan
| |
Collapse
|
46
|
das Graças José Ventura V, Pereira PD, Pires MC, Asevedo AA, de Oliveira Jorge A, Dos Santos ACP, de Moura Costa AS, Dos Reis Gomes AG, Lima BF, Pessoa BP, Cimini CCR, de Andrade CMV, Ponce D, Rios DRA, Pereira EC, Manenti ERF, de Almeida Cenci EP, Costa FR, Anschau F, Aranha FG, Vigil FMB, Bartolazzi F, Aguiar GG, Grizende GMS, Batista JDL, Neves JVB, Ruschel KB, do Nascimento L, de Oliveira LMC, Kopittke L, de Castro LC, Sacioto MF, Carneiro M, Gonçalves MA, Bicalho MAC, da Paula Sordi MA, da Cunha Severino Sampaio N, Paraíso PG, Menezes RM, Araújo SF, de Assis VCM, de Paula Farah K, Marcolino MS. Temporal validation of the MMCD score to predict kidney replacement therapy and in-hospital mortality in COVID-19 patients. BMC Nephrol 2023; 24:292. [PMID: 37794354 PMCID: PMC10552198 DOI: 10.1186/s12882-023-03341-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2023] [Accepted: 09/20/2023] [Indexed: 10/06/2023] Open
Abstract
BACKGROUND Acute kidney injury has been described as a common complication in patients hospitalized with COVID-19, which may lead to the need for kidney replacement therapy (KRT) in its most severe forms. Our group developed and validated the MMCD score in Brazilian COVID-19 patients to predict KRT, which showed excellent performance using data from 2020. This study aimed to validate the MMCD score in a large cohort of patients hospitalized with COVID-19 in a different pandemic phase and assess its performance to predict in-hospital mortality. METHODS This study is part of the "Brazilian COVID-19 Registry", a retrospective observational cohort of consecutive patients hospitalized for laboratory-confirmed COVID-19 in 25 Brazilian hospitals between March 2021 and August 2022. The primary outcome was KRT during hospitalization and the secondary was in-hospital mortality. We also searched literature for other prediction models for KRT, to assess the results in our database. Performance was assessed using area under the receiving operator characteristic curve (AUROC) and the Brier score. RESULTS A total of 9422 patients were included, 53.8% were men, with a median age of 59 (IQR 48-70) years old. The incidence of KRT was 8.8% and in-hospital mortality was 18.1%. The MMCD score had excellent discrimination and overall performance to predict KRT (AUROC: 0.916 [95% CI 0.909-0.924]; Brier score = 0.057). Despite the excellent discrimination and overall performance (AUROC: 0.922 [95% CI 0.914-0.929]; Brier score = 0.100), the calibration was not satisfactory concerning in-hospital mortality. A random forest model was applied in the database, with inferior performance to predict KRT requirement (AUROC: 0.71 [95% CI 0.69-0.73]). CONCLUSION The MMCD score is not appropriate for in-hospital mortality but demonstrates an excellent predictive ability to predict KRT in COVID-19 patients. The instrument is low cost, objective, fast and accurate, and can contribute to supporting clinical decisions in the efficient allocation of assistance resources in patients with COVID-19.
Collapse
Affiliation(s)
- Vanessa das Graças José Ventura
- Medical School and University Hospital, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 190, Belo Horizonte, Brazil.
| | - Polianna Delfino Pereira
- Department of Internal Medicine, Medical School, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 190, Belo Horizonte, Brazil
- Institute for Health Technology Assessment (IATS/ CNPq), R. Ramiro Barcelos, 2359, Porto Alegre, Brazil
| | - Magda Carvalho Pires
- Department of Statistics, Universidade Federal de Minas Gerais, Av. Presidente Antônio Carlos, 6627, Belo Horizonte, Brazil
| | - Alisson Alves Asevedo
- Universidade Federal Dos Vales Do Jequitinhonha E Mucuri (UFVJM), R. Cruzeiro, 01. , Teófilo Otoni, Minas Gerais, Brazil
| | - Alzira de Oliveira Jorge
- Medical School and University Hospital, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 190, Belo Horizonte, Brazil
- Hospital Risoleta Tolentino Neves, R. das Gabirobas, 01, Belo Horizonte, Brazil
| | | | | | | | - Beatriz Figueiredo Lima
- Medical School and University Hospital, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 190, Belo Horizonte, Brazil
- Hospital Metropolitano Odilon Behrens, R. Formiga, 50, Belo Horizonte, Brazil
| | - Bruno Porto Pessoa
- Hospital Júlia Kubitschek, Av. Professor Alfredo Balena, 190, Belo Horizonte, Brazil
| | - Christiane Corrêa Rodrigues Cimini
- Universidade Federal Dos Vales Do Jequitinhonha E Mucuri (UFVJM), R. Cruzeiro, 01. , Teófilo Otoni, Minas Gerais, Brazil
- Hospital Santa Rosália, R. Do Cruzeiro, 01, Teófilo Otoni, Brazil
| | | | - Daniela Ponce
- Botucatu Medical School, Universidade Estadual Paulista "Júlio de Mesquita Filho", Av. Prof. Mário Rubens Guimarães Montenegro, Botucatu, Brazil
| | | | | | | | | | | | - Fernando Anschau
- Hospital Nossa Senhora da Conceição, Av. Francisco Trein, 326, Porto Alegre, Brazil
| | | | | | - Frederico Bartolazzi
- Hospital Santo Antônio, Pç. Dr. Márcio Carvalho Lopes Filho, 501, Curvelo, Brazil
| | - Gabriella Genta Aguiar
- Universidade José Do Rosário Vellano (UNIFENAS), R. Boaventura, 50, Belo Horizonte, Brazil
| | | | - Joanna d'Arc Lyra Batista
- Institute for Health Technology Assessment (IATS/ CNPq), R. Ramiro Barcelos, 2359, Porto Alegre, Brazil
- Medical School, Universidade Federal da Fronteira Sul, SC-484 Km 02, Chapecó, Brazil
| | - João Victor Baroni Neves
- Faculdade de Ciências Médicas de Minas Gerais, Al. Ezequiel Dias, 275, Belo Horizonte, Minas Gerais, Brazil
| | | | - Letícia do Nascimento
- Hospital Universitário de Santa Maria, Av. Roraima, 1000, Prédio 22, Santa Maria, Brazil
| | | | - Luciane Kopittke
- Hospital Nossa Senhora da Conceição, Av. Francisco Trein, 326, Porto Alegre, Brazil
| | | | - Manuela Furtado Sacioto
- Faculdade de Ciências Médicas de Minas Gerais, Al. Ezequiel Dias, 275, Belo Horizonte, Minas Gerais, Brazil
| | - Marcelo Carneiro
- Hospital Santa Cruz, R. Fernando Abott, 174, Santa Cruz Do Sul, Brazil
| | - Marcos André Gonçalves
- Computer Science Department, Universidade Federal de Minas Gerais, Av. Presidente Antônio Carlos, 6627, Belo Horizonte, Brazil
| | - Maria Aparecida Camargos Bicalho
- Medical School and University Hospital, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 190, Belo Horizonte, Brazil
- Hospital João XXIII, Av. Professor Alfredo Balena, 400, Belo Horizonte, Brazil
| | - Mônica Aparecida da Paula Sordi
- Botucatu Medical School, Universidade Estadual Paulista "Júlio de Mesquita Filho", Av. Prof. Mário Rubens Guimarães Montenegro, Botucatu, Brazil
| | | | - Pedro Gibson Paraíso
- Orizonti Instituto de Saúde E Longevidade, Av. José Do Patrocínio Pontes, 1355, Belo Horizonte, Brazil
| | | | | | | | - Katia de Paula Farah
- Medical School and University Hospital, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 190, Belo Horizonte, Brazil
| | - Milena Soriano Marcolino
- Medical School and University Hospital, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 190, Belo Horizonte, Brazil
- Department of Internal Medicine, Medical School, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 190, Belo Horizonte, Brazil
- Institute for Health Technology Assessment (IATS/ CNPq), R. Ramiro Barcelos, 2359, Porto Alegre, Brazil
- Telehealth Center, University Hospital, Universidade Federal de Minas Gerais, Av. Professor Alfredo Balena, 110, Belo Horizonte, Brazil
| |
Collapse
|
47
|
Reyes-Esteves S, Kumar M, Kasner SE, Witsch J. Clinical Grading Scales and Neuroprognostication in Acute Brain Injury. Semin Neurol 2023; 43:664-674. [PMID: 37788680 DOI: 10.1055/s-0043-1775749] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/05/2023]
Abstract
Prediction of neurological clinical outcome after acute brain injury is critical because it helps guide discussions with patients and families and informs treatment plans and allocation of resources. Numerous clinical grading scales have been published that aim to support prognostication after acute brain injury. However, the development and validation of clinical scales lack a standardized approach. This in turn makes it difficult for clinicians to rely on prognostic grading scales and to integrate them into clinical practice. In this review, we discuss quality measures of score development and validation and summarize available scales to prognosticate outcomes after acute brain injury. These include scales developed for patients with coma, cardiac arrest, ischemic stroke, nontraumatic intracerebral hemorrhage, subarachnoid hemorrhage, and traumatic brain injury; for each scale, we discuss available validation studies.
Collapse
Affiliation(s)
- Sahily Reyes-Esteves
- Department of Neurology, Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania
| | - Monisha Kumar
- Department of Neurology, Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania
| | - Scott E Kasner
- Department of Neurology, Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania
| | - Jens Witsch
- Department of Neurology, Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania
| |
Collapse
|
48
|
Liao H, Xu Y, Meng Q, Mao Z, Qiao Y, Liu Y, Zheng Q. A convolutional neural network-based, quantitative complete blood count scattergram-mapping framework promptly screens acute promyelocytic leukemia with high sensitivity. Cancer 2023; 129:2986-2998. [PMID: 37254628 DOI: 10.1002/cncr.34890] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Revised: 05/09/2023] [Accepted: 05/10/2023] [Indexed: 06/01/2023]
Abstract
BACKGROUND Acute promyelocytic leukemia (APL) is a subtype of acute myeloid leukemia (AML) characterized by its rapidly progressive and fatal clinical course if untreated, although it is curable if treated in a timely manner. Promptly screening patients who have results that are suspicious for APL is vital to overcome early death. METHODS The authors developed an innovative framework consisting of ResNet-18, a convolutional neural network architecture, with the objective of quantitatively mapping a complete blood count (CBC) scattergram to quickly and robustly indicate a probable susceptibility to APL. Three hundred and twenty scattergrams of the white blood cell differential channel from 51 patients with APL, 510 scattergrams from 105 patients who had non-APL AML, and 320 scattergrams from 320 healthy controls were randomly stratified at a ratio of 4:1 and split into training and testing data sets to accomplish five-fold cross-validation. RESULTS Both the area under the curve and the average precision of >0.99 were achieved in each fold. Three hundred four of the 320 APL scattergrams (95%) were correctly flagged by the model, which outcompeted the CBC review rules recommended by the International Society of Laboratory Hematology (all p < .001). External validation based on an independent testing data set that included 56 scattergrams from 31 patients with APL, 56 scattergrams from 55 patients with non-APL AML, and 64 scattergrams from 64 healthy controls also confirmed the sensitivity and specificity of the framework. CONCLUSIONS To the authors' knowledge, their convolutional neural network-based framework is the first to use scattergram output from routine CBC analysis to map suspicious APL early with outstanding sensitivity, specificity, and precision. The authors also describe a new CBC workflow incorporating this framework upstream of the morphologic review, which would provide the earliest flag for APL. PLAIN LANGUAGE SUMMARY The authors propose an innovative way to visualize complete blood counts (CBCs) by mapping the difference in white blood cell counts using automated CBC analysis to identify potential acute promyelocytic leukemia (APL) using a convolutional neural network (CNN), which can eliminate the potential pitfalls of manual observation. Analyses of an unprecedented, realistic data set validated that the quantitative relationship between the CBC scattergram and an APL abnormality is highly consistent. This is the first study to date focusing on screening for APL using scattergrams of the difference in white blood cell counts from routine CBC tests and has significant clinical relevance. The authors recommend using this method even before analyzing cell images, which could provide the earliest way to screen for APL in a sensitive and accurate way.
Collapse
Affiliation(s)
- Hongyan Liao
- Department of Laboratory Medicine, West China Hospital, Sichuan University, Chengdu, China
| | - Yuanxin Xu
- College of Electrical Engineering, Sichuan University, Chengdu, China
| | - Qiang Meng
- Department of Laboratory Medicine, West China Hospital, Sichuan University, Chengdu, China
| | - Zhigang Mao
- Department of Laboratory Medicine, West China Hospital, Sichuan University, Chengdu, China
| | - Yifan Qiao
- College of Computer Science, Sichuan University, Chengdu, China
| | - Yan Liu
- College of Electrical Engineering, Sichuan University, Chengdu, China
| | - Qin Zheng
- Department of Laboratory Medicine, West China Hospital, Sichuan University, Chengdu, China
- Med-X Center for Informatics, Sichuan University, Chengdu, China
| |
Collapse
|
49
|
Lapi F, Bianchini E, Marconi E, Medea G, Piccinni C, Maggioni AP, Dondi L, Pedrini A, Martini N, Cricelli C. A methodology to assess the population size and estimate the needed resources for new licensed medications by combining clinical and administrative databases: The example of glycated haemoglobin in type 2 diabetes. Pharmacoepidemiol Drug Saf 2023; 32:1083-1092. [PMID: 37208842 DOI: 10.1002/pds.5641] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2022] [Revised: 05/10/2023] [Accepted: 05/15/2023] [Indexed: 05/21/2023]
Abstract
PURPOSE To develop and validate a model to estimate glycated haemoglobin (HbA1c) values in patients with type 2 diabetes mellitus (T2DM) using a clinical data source, with the aim to apply this equation to administrative databases. METHODS Using a primary care and administrative Italian databases, namely the Health Search database (HSD) and the ReS (Ricerca e Salute) database, we selected all patients aged 18 years or older on 31 December 2018 being diagnosed with T2DM and without prior prescription of sodium-glucose cotransporter-2 (SGLT-2) inhibitors. We included patients prescribed with and adherent to metformin. HSD was used to develop and test (using 2019 data as well) the algorithm imputing HbA1c values ≥7% according to a series of covariates. The algorithm was gathered by combining beta-coefficients being estimated by logistic regression models using complete case (excluding missing values) and imputed (after multiple imputation) dataset. The final algorithm was applied to ReS database using the same covariates. RESULTS The tested algorithms were able to explain 17%-18% variation in assessing HbA1c values. Good discrimination (70%) and calibration were obtained as well. The best algorithm (three) cut-offs, namely those providing correct classifications ranging 66%-70% was therefore calculated and applied to ReS database. By doing so, from 52 999 (27.9, 95% CI: 27.7%-28.1%) to 74 250 (40.1%, 95% CI: 38.9%-39.3%) patients were estimated with HbA1c ≥7%. CONCLUSION Through this methodology, healthcare authorities should be able to quantify the population eligible to a new licensed medication, such as SGLT-2 inhibitors, and to simulate scenarios to assess reimbursement criteria according to precise estimates.
Collapse
Affiliation(s)
- Francesco Lapi
- Health Search, Italian College of General Practitioners and Primary Care, Florence, Italy
| | - Elisa Bianchini
- Health Search, Italian College of General Practitioners and Primary Care, Florence, Italy
| | - Ettore Marconi
- Health Search, Italian College of General Practitioners and Primary Care, Florence, Italy
| | - Gerardo Medea
- Italian College of General Practitioners and Primary Care, Florence, Italy
| | - Carlo Piccinni
- Fondazione ReS (Ricerca e Salute - Health and Research Foundation), Rome, Italy
| | - Aldo P Maggioni
- Fondazione ReS (Ricerca e Salute - Health and Research Foundation), Rome, Italy
- ANMCO Research Center Heart Care Foundation, Firenze, Italy
| | - Letizia Dondi
- Fondazione ReS (Ricerca e Salute - Health and Research Foundation), Rome, Italy
| | - Antonella Pedrini
- Fondazione ReS (Ricerca e Salute - Health and Research Foundation), Rome, Italy
| | - Nello Martini
- Fondazione ReS (Ricerca e Salute - Health and Research Foundation), Rome, Italy
| | - Claudio Cricelli
- Italian College of General Practitioners and Primary Care, Florence, Italy
| |
Collapse
|
50
|
Mari T, Henderson J, Ali SH, Hewitt D, Brown C, Stancak A, Fallon N. Machine learning and EEG can classify passive viewing of discrete categories of visual stimuli but not the observation of pain. BMC Neurosci 2023; 24:50. [PMID: 37715119 PMCID: PMC10504739 DOI: 10.1186/s12868-023-00819-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2023] [Accepted: 09/06/2023] [Indexed: 09/17/2023] Open
Abstract
Previous studies have demonstrated the potential of machine learning (ML) in classifying physical pain from non-pain states using electroencephalographic (EEG) data. However, the application of ML to EEG data to categorise the observation of pain versus non-pain images of human facial expressions or scenes depicting pain being inflicted has not been explored. The present study aimed to address this by training Random Forest (RF) models on cortical event-related potentials (ERPs) recorded while participants passively viewed faces displaying either pain or neutral expressions, as well as action scenes depicting pain or matched non-pain (neutral) scenarios. Ninety-one participants were recruited across three samples, which included a model development group (n = 40) and a cross-subject validation group (n = 51). Additionally, 25 participants from the model development group completed a second experimental session, providing a within-subject temporal validation sample. The analysis of ERPs revealed an enhanced N170 component in response to faces compared to action scenes. Moreover, an increased late positive potential (LPP) was observed during the viewing of pain scenes compared to neutral scenes. Additionally, an enhanced P3 response was found when participants viewed faces displaying pain expressions compared to neutral expressions. Subsequently, three RF models were developed to classify images into faces and scenes, neutral and pain scenes, and neutral and pain expressions. The RF model achieved classification accuracies of 75%, 64%, and 69% for cross-validation, cross-subject, and within-subject classifications, respectively, along with reasonably calibrated predictions for the classification of face versus scene images. However, the RF model was unable to classify pain versus neutral stimuli above chance levels when presented with subsequent tasks involving images from either category. These results expand upon previous findings by externally validating the use of ML in classifying ERPs related to different categories of visual images, namely faces and scenes. The results also indicate the limitations of ML in distinguishing pain and non-pain connotations using ERP responses to the passive viewing of visually similar images.
Collapse
Affiliation(s)
- Tyler Mari
- Department of Psychology, Institute of Population Health, University of Liverpool, 2.21 Eleanor Rathbone Building, Bedford Street South, Liverpool, L69 7ZA, UK.
| | - Jessica Henderson
- Department of Psychology, Institute of Population Health, University of Liverpool, 2.21 Eleanor Rathbone Building, Bedford Street South, Liverpool, L69 7ZA, UK
| | - S Hasan Ali
- Department of Psychology, Institute of Population Health, University of Liverpool, 2.21 Eleanor Rathbone Building, Bedford Street South, Liverpool, L69 7ZA, UK
| | - Danielle Hewitt
- Department of Psychology, Institute of Population Health, University of Liverpool, 2.21 Eleanor Rathbone Building, Bedford Street South, Liverpool, L69 7ZA, UK
| | - Christopher Brown
- Department of Psychology, Institute of Population Health, University of Liverpool, 2.21 Eleanor Rathbone Building, Bedford Street South, Liverpool, L69 7ZA, UK
| | - Andrej Stancak
- Department of Psychology, Institute of Population Health, University of Liverpool, 2.21 Eleanor Rathbone Building, Bedford Street South, Liverpool, L69 7ZA, UK
| | - Nicholas Fallon
- Department of Psychology, Institute of Population Health, University of Liverpool, 2.21 Eleanor Rathbone Building, Bedford Street South, Liverpool, L69 7ZA, UK
| |
Collapse
|