1
|
Li Q, Lv H, Chen Y, Shen J, Shi J, Zhou C, Yan F. Development and validation of a machine learning prediction model for perioperative red blood cell transfusions in cardiac surgery. Int J Med Inform 2024; 184:105343. [PMID: 38286086 DOI: 10.1016/j.ijmedinf.2024.105343] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Revised: 01/10/2024] [Accepted: 01/17/2024] [Indexed: 01/31/2024]
Abstract
OBJECTIVE Several machine learning (ML) models have been used in perioperative red blood cell (RBC) transfusion risk for cardiac surgery with limited generalizability and no external validation. Hence, we sought to develop and comprehensively externally validate a ML model in a large dataset to estimate RBC transfusion in cardiac surgery with cardiopulmonary bypass (CPB). DESIGN A retrospective analysis of a multicenter clinical trial (NCT03782350). PATIENTS The study patients who underwent cardiac surgery with CPB came from four cardiac centers in China and Medical Information Mart for Intensive Cared (MIMIC-IV) dataset. MEASUREMENTS Data from Fuwai Hospital were used to develop an individualized prediction model for RBC transfusion. The model was externally validated in the data from three other centers and MIMIC-IV dataset. Twelve models were constructed. MAIN RESULTS A total of 11,201 eligible patients were included in the model development (2420 in Fuwai Hospital) and external validation (563 in the other three centers and 8218 in the MIMIC-IV dataset). A significant difference was observed between the Logistic Regression and CatboostClassifier (0.72 Vs. 0.74, P = 0.031) or RandomForestClassifier (0.72 Vs. 0.75 p = 0.012) in the external validation and MIMIV-IV datasets (age ≤ 70:0.63 Vs. 0.71, p < 0.001; age > 70:0.63 Vs. 0.70, 0.63 Vs. 0.71, p < 0.001). The CatboostClassifier and RandomForestClassifier model was comparable in development (0.83 Vs. 0.82, p = 0.419), external (0.74 Vs. 0.75, p = 0.268), and MIMIC-IV datasets (age ≤ 70: 0.71 Vs. 0.71, p = 0.574; age > 70: 0.70 Vs. 0.71, p = 0.981). Of note, they outperformed other ML models with excellent discrimination and calibration. The CatboostClassifier and RandomForestClassifier models achieved higher area under precision-recall curve and lower brier loss score in validation and MIMIC-IV datasets. Additionally, we confirmed that low preoperative hemoglobin, low body mass index, old age, and female sex increased the risk of RBC transfusion. CONCLUSIONS In our study, enrolling a broad range of cardiovascular surgeries with CPB and utilizing a restrictive RBC transfusion strategy, robustly validates the generalizability of ML algorithms for predicting RBC transfusion risk. Notably, the CatboostClassifier and RandomForestClassifier exhibit strong external clinical applicability, underscoring their potential for widespread adoption. This study provides compelling evidence supporting the efficacy and practical value of ML-based approaches in enhancing transfusion risk prediction in clinical practice.
Collapse
Affiliation(s)
- Qian Li
- Department of Anesthesiology, Fuwai Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College/National Center for Cardiovascular Diseases, Beijing 100037, China
| | - Hong Lv
- Department of Anesthesiology, Fuwai Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College/National Center for Cardiovascular Diseases, Beijing 100037, China
| | - Yuye Chen
- Department of Anesthesiology, Fuwai Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College/National Center for Cardiovascular Diseases, Beijing 100037, China
| | - Jingjia Shen
- Department of Anesthesiology, Fuwai Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College/National Center for Cardiovascular Diseases, Beijing 100037, China
| | - Jia Shi
- Department of Anesthesiology, Fuwai Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College/National Center for Cardiovascular Diseases, Beijing 100037, China
| | - Chenghui Zhou
- Department of Anesthesiology, Fuwai Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College/National Center for Cardiovascular Diseases, Beijing 100037, China; Center for Anesthesiology, Beijing Anzhen Hospital, Capital Medical University, Beijing, 100029, China.
| | - Fuxia Yan
- Department of Anesthesiology, Fuwai Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College/National Center for Cardiovascular Diseases, Beijing 100037, China.
| |
Collapse
|
2
|
Zhang Q, Gao Y, Tian Y, Gao S, Diao X, Ji H, Wang Y, Ji B. A transfusion risk stratification score to facilitate quality management in cardiopulmonary bypass. Transfusion 2023; 63:1495-1505. [PMID: 37458390 DOI: 10.1111/trf.17487] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2023] [Revised: 05/19/2023] [Accepted: 06/09/2023] [Indexed: 08/12/2023]
Abstract
BACKGROUND Our previous showed that a blood management program in the cardiopulmonary bypass (CPB) department, reduced red blood cell (RBC) transfusion and complications, but assessing transfusion practice solely based on transfusion rates was insufficient. This study aimed to design a risk stratification score to predict perioperative RBC transfusion to guide targeted measures for on-pump cardiac surgery patients. STUDY DESIGN AND METHODS We analyzed data from 42,435 adult cardiac patients. Eight predictors were entered into the final model including age, sex, anemia, New York Heart Association classification, body surface area, cardiac surgery history, emergency surgery, and surgery type. We then simplified the score to an integer-based system. The area under the receiver operating characteristic curve (AUC), Hosmer-Lemeshow goodness-of-fit test, and a calibration curve were used for its performance test. The score was compared to existing scores. RESULTS The final score included eight predictors. The AUC for the model was 0.77 (95% CI, 0.76-0.77) and 0.77 (95% CI, 0.76-0.78) in the training and test set, respectively. The calibration curves showed a good fit. The risk score was finally grouped into low-risk (score of 0-13 points), medium-risk (14-19 points), and high-risk (more than 19 points). The score had better predictive power compared to the other two existing risk scores. DISCUSSION We developed an effective risk stratification score with eight variables to predict perioperative RBC transfusion for on-pump cardiac surgery. It assists perfusionists in proactively preparing blood conservation measures for high-risk patients before surgery.
Collapse
Affiliation(s)
- Qiaoni Zhang
- Department of Cardiopulmonary Bypass, Fuwai Hospital, Chinese Academy of Medical Science and Peking Union Medical College, National Clinical Research Center for Cardiovascular Disease, Beijing, China
| | - Yuchen Gao
- Department of Anesthesiology, Fuwai Hospital, Chinese Academy of Medical Science and Peking Union Medical College, National Clinical Research Center for Cardiovascular Disease, Beijing, China
| | - Yu Tian
- Department of Anesthesiology, Beijing Anzhen Hospital, Capital Medical University, Beijing, China
| | - Sizhe Gao
- Department of Cardiopulmonary Bypass, Fuwai Hospital, Chinese Academy of Medical Science and Peking Union Medical College, National Clinical Research Center for Cardiovascular Disease, Beijing, China
| | - Xiaolin Diao
- Department of Information Center, Fuwai Hospital, Chinese Academy of Medical Science and Peking Union Medical College, National Clinical Research Center for Cardiovascular Disease, Beijing, China
| | - Hongwen Ji
- Department of Anesthesiology, Fuwai Hospital, Chinese Academy of Medical Science and Peking Union Medical College, National Clinical Research Center for Cardiovascular Disease, Beijing, China
| | - Yuefu Wang
- Surgery Intensive Care Unit & Center of Anesthesia, Beijing Shijitan Hospital, Capital Medical University, Beijing, China
| | - Bingyang Ji
- Department of Cardiopulmonary Bypass, Fuwai Hospital, Chinese Academy of Medical Science and Peking Union Medical College, National Clinical Research Center for Cardiovascular Disease, Beijing, China
| |
Collapse
|
3
|
Dhiman P, Ma J, Gibbs VN, Rampotas A, Kamal H, Arshad SS, Kirtley S, Doree C, Murphy MF, Collins GS, Palmer AJR. Systematic review highlights high risk of bias of clinical prediction models for blood transfusion in patients undergoing elective surgery. J Clin Epidemiol 2023; 159:10-30. [PMID: 37156342 DOI: 10.1016/j.jclinepi.2023.05.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2022] [Revised: 04/21/2023] [Accepted: 05/01/2023] [Indexed: 05/10/2023]
Abstract
BACKGROUND Blood transfusion can be a lifesaving intervention after perioperative blood loss. Many prediction models have been developed to identify patients most likely to require blood transfusion during elective surgery, but it is unclear whether any are suitable for clinical practice. STUDY DESIGN AND SETTING We conducted a systematic review, searching MEDLINE, Embase, PubMed, The Cochrane Library, Transfusion Evidence Library, Scopus, and Web of Science databases for studies reporting the development or validation of a blood transfusion prediction model in elective surgery patients between January 1, 2000 and June 30, 2021. We extracted study characteristics, discrimination performance (c-statistics) of final models, and data, which we used to perform risk of bias assessment using the Prediction model risk of bias assessment tool (PROBAST). RESULTS We reviewed 66 studies (72 developed and 48 externally validated models). Pooled c-statistics of externally validated models ranged from 0.67 to 0.78. Most developed and validated models were at high risk of bias due to handling of predictors, validation methods, and too small sample sizes. CONCLUSION Most blood transfusion prediction models are at high risk of bias and suffer from poor reporting and methodological quality, which must be addressed before they can be safely used in clinical practice.
Collapse
Affiliation(s)
- Paula Dhiman
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK; NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK.
| | - Jie Ma
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Victoria N Gibbs
- Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Alexandros Rampotas
- Systematic Review Initiative, NHS Blood & Transplant, John Radcliffe Hospital, Oxford, UK
| | - Hassan Kamal
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK; School of Medicine, University of Dundee, Ninewells Hospital & Medical School, Dundee, Scotland DD1 9SY
| | - Sahar S Arshad
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Shona Kirtley
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Carolyn Doree
- Systematic Review Initiative, NHS Blood & Transplant, John Radcliffe Hospital, Oxford, UK
| | - Michael F Murphy
- NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK; Systematic Review Initiative, NHS Blood & Transplant, John Radcliffe Hospital, Oxford, UK; NIHR Blood and Transplant Research Unit in Data Driven Transfusion Practice, Nuffield Division of Clinical Laboratory Sciences, Radcliffe Department of Medicine, University of Oxford, Oxford, UK
| | - Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK; NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Antony J R Palmer
- Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford, UK; NIHR Blood and Transplant Research Unit in Data Driven Transfusion Practice, Nuffield Division of Clinical Laboratory Sciences, Radcliffe Department of Medicine, University of Oxford, Oxford, UK; Oxford University Hospitals, Nuffield Orthopaedic Centre, Windmill Road, Headington, Oxford OX3 7HE, UK
| |
Collapse
|