1
|
Ji J, Zhang T, Zhu L, Yao Y, Mei J, Sun L, Zhang G. Using machine learning to develop preoperative model for lymph node metastasis in patients with bladder urothelial carcinoma. BMC Cancer 2024; 24:725. [PMID: 38872141 PMCID: PMC11170799 DOI: 10.1186/s12885-024-12467-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2023] [Accepted: 06/03/2024] [Indexed: 06/15/2024] Open
Abstract
BACKGROUND Lymph node metastasis (LNM) is associated with worse prognosis in bladder urothelial carcinoma (BUC) patients. This study aimed to develop and validate machine learning (ML) models to preoperatively predict LNM in BUC patients treated with radical cystectomy (RC). METHODS We retrospectively collected demographic, pathological, imaging, and laboratory information of BUC patients who underwent RC and bilateral lymphadenectomy in our institution. Patients were randomly categorized into training set and testing set. Five ML algorithms were utilized to establish prediction models. The performance of each model was assessed by the area under the receiver operating characteristic curve (AUC) and accuracy. Finally, we calculated the corresponding variable coefficients based on the optimal model to reveal the contribution of each variable to LNM. RESULTS A total of 524 and 131 BUC patients were finally enrolled into training set and testing set, respectively. We identified that the support vector machine (SVM) model had the best prediction ability with an AUC of 0.934 (95% confidence interval [CI]: 0.903-0.964) and accuracy of 0.916 in the training set, and an AUC of 0.855 (95%CI: 0.777-0.933) and accuracy of 0.809 in the testing set. The SVM model contained 14 predictors, and positive lymph node in imaging contributed the most to the prediction of LNM in BUC patients. CONCLUSIONS We developed and validated the ML models to preoperatively predict LNM in BUC patients treated with RC, and identified that the SVM model with 14 variables had the best performance and high levels of clinical applicability.
Collapse
Affiliation(s)
- Junjie Ji
- Department of Urology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Tianwei Zhang
- Department of Urology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Ling Zhu
- Shandong Key Laboratory of Digital Medicine and Computer Assisted Surgery, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Yu Yao
- Department of Urology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Jingchang Mei
- Department of Urology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Lijiang Sun
- Department of Urology, The Affiliated Hospital of Qingdao University, Qingdao, China
| | - Guiming Zhang
- Department of Urology, The Affiliated Hospital of Qingdao University, Qingdao, China.
| |
Collapse
|
2
|
Jiang T, Nagy D, Rosellini AJ, Horváth-Puhó E, Keyes KM, Lash TL, Galea S, Sørensen HT, Gradus JL. Prediction of suicide attempts among persons with depression: a population-based case cohort study. Am J Epidemiol 2024; 193:827-834. [PMID: 38055633 DOI: 10.1093/aje/kwad237] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2023] [Revised: 11/17/2023] [Accepted: 12/01/2023] [Indexed: 12/08/2023] Open
Abstract
Studies have highlighted the potential importance of modeling interactions for suicide attempt prediction. This case-cohort study identified risk factors for suicide attempts among persons with depression in Denmark using statistical approaches that do (random forests) or do not (least absolute shrinkage and selection operator regression [LASSO]) model interactions. Cases made a nonfatal suicide attempt (n = 6032) between 1995 and 2015. The comparison subcohort was a 5% random sample of all persons in Denmark on January 1, 1995 (n = 11 963). We used random forests and LASSO for sex-stratified prediction of suicide attempts from demographic variables, psychiatric and somatic diagnoses, and treatments. Poisonings, psychiatric disorders, and medications were important predictors for both sexes. Area under the receiver-operating characteristic curve (AUC) values were higher in LASSO models (in men, 0.85, 95% CI, 0.84-0.86; in women, 0.89, 95% C, 0.88-0.90) than random forests (in men, 0.76, 95% CI, 0.74-0.78; in women, 0.79, 95% CI = 0.78-0.81). Automatic detection of interactions via random forests did not result in better model performance than LASSO models that did not model interactions. Due to the complex nature of psychiatric comorbidity and suicide, modeling interactions may not always be the optimal statistical approach to enhancing suicide attempt prediction in high-risk samples. This article is part of a Special Collection on Mental Health.
Collapse
Affiliation(s)
- Tammy Jiang
- Department of Epidemiology, Boston University School of Public Health, Boston, MA 02118, United States
| | - Dávid Nagy
- Department of Clinical Epidemiology, Aarhus University Hospital and Aarhus University, 8200 Aarhus N, Denmark
| | - Anthony J Rosellini
- Department of Epidemiology, Boston University School of Public Health, Boston, MA 02118, United States
- Center for Anxiety and Related Disorders, Department of Psychological and Brain Sciences, Boston University, Boston, MA 02215, United States
| | - Erzsébet Horváth-Puhó
- Department of Clinical Epidemiology, Aarhus University Hospital and Aarhus University, 8200 Aarhus N, Denmark
| | - Katherine M Keyes
- Department of Epidemiology, Columbia University Mailman School of Public Health, New York, NY 10032, United States
| | - Timothy L Lash
- Department of Clinical Epidemiology, Aarhus University Hospital and Aarhus University, 8200 Aarhus N, Denmark
- Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA 30322, United States
| | - Sandro Galea
- Department of Epidemiology, Boston University School of Public Health, Boston, MA 02118, United States
- Department of Family Medicine, Boston University School of Medicine, Boston, MA 02118, United States
| | - Henrik T Sørensen
- Department of Epidemiology, Boston University School of Public Health, Boston, MA 02118, United States
- Department of Clinical Epidemiology, Aarhus University Hospital and Aarhus University, 8200 Aarhus N, Denmark
| | - Jaimie L Gradus
- Department of Epidemiology, Boston University School of Public Health, Boston, MA 02118, United States
- Department of Clinical Epidemiology, Aarhus University Hospital and Aarhus University, 8200 Aarhus N, Denmark
- Department of Psychiatry, Boston University School of Medicine, Boston, MA 02118, United States
| |
Collapse
|
3
|
Clift AK, Tan PS, Patone M, Liao W, Coupland C, Bashford-Rogers R, Sivakumar S, Hippisley-Cox J. Predicting the risk of pancreatic cancer in adults with new-onset diabetes: development and internal-external validation of a clinical risk prediction model. Br J Cancer 2024; 130:1969-1978. [PMID: 38702436 PMCID: PMC11183048 DOI: 10.1038/s41416-024-02693-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Revised: 04/08/2024] [Accepted: 04/11/2024] [Indexed: 05/06/2024] Open
Abstract
BACKGROUND The National Institute for Health and Care Excellence (NICE) recommends that people aged 60+ years with newly diagnosed diabetes and weight loss undergo abdominal imaging to assess for pancreatic cancer. More nuanced stratification could lead to enrichment of these referral pathways. METHODS Population-based cohort study of adults aged 30-85 years at type 2 diabetes diagnosis (2010-2021) using the QResearch primary care database in England linked to secondary care data, the national cancer registry and mortality registers. Clinical prediction models were developed to estimate risks of pancreatic cancer diagnosis within 2 years and evaluated using internal-external cross-validation. RESULTS Seven hundred and sixty-seven of 253,766 individuals were diagnosed with pancreatic cancer within 2 years. Models included age, sex, BMI, prior venous thromboembolism, digoxin prescription, HbA1c, ALT, creatinine, haemoglobin, platelet count; and the presence of abdominal pain, weight loss, jaundice, heartburn, indigestion or nausea (previous 6 months). The Cox model had the highest discrimination (Harrell's C-index 0.802 (95% CI: 0.797-0.817)), the highest clinical utility, and was well calibrated. The model's highest 1% of predicted risks captured 12.51% of pancreatic cancer cases. NICE guidance had 3.95% sensitivity. DISCUSSION A new prediction model could have clinical utility in identifying individuals with recent onset diabetes suitable for fast-track abdominal imaging.
Collapse
Affiliation(s)
- Ash Kieran Clift
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
- Cancer Research UK Oxford Centre, University of Oxford, Oxford, UK
| | - Pui San Tan
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Martina Patone
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Weiqi Liao
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Carol Coupland
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
- Centre for Academic Primary Care, School of Medicine, University of Nottingham, Nottingham, UK
| | - Rachael Bashford-Rogers
- Wellcome Centre for Human Genetics, University of Oxford, Oxford, UK
- Department of Biochemistry, University of Oxford, Oxford, UK
| | - Shivan Sivakumar
- Institute of Immunology and Immunotherapy, Birmingham Medical School, Birmingham, UK
- Cancer Centre, Queen Elizabeth Hospital, University Hospitals of Birmingham NHS Trust, Birmingham, UK
| | - Julia Hippisley-Cox
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK.
| |
Collapse
|
4
|
Vargas KG, Rütten T, Siemes B, Brockmeyer M, Parco C, Hoss A, Schlesinger S, Jung C, Roden M, Kelm M, Wolff G, Kuss O. Assessing the potential for precision medicine in body weight reduction with regard to type 2 diabetes mellitus therapies: A meta-regression analysis of 120 randomized controlled trials. Diabetes Obes Metab 2024; 26:2139-2146. [PMID: 38425176 DOI: 10.1111/dom.15519] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/23/2023] [Revised: 01/25/2024] [Accepted: 02/05/2024] [Indexed: 03/02/2024]
Abstract
AIMS To assess the potential for precision medicine in type 2 diabetes by quantifying the variability of body weight as response to pharmacological treatment and to identify predictors which could explain this variability. METHODS We used randomized clinical trials (RCTs) comparing glucose-lowering drugs (including but not limited to sodium-glucose cotransporter-2 inhibitors, glucagon-like peptide-1 receptor agonists and thiazolidinediones) to placebo from four recent systematic reviews. RCTs reporting on body weight after treatment to allow for calculation of its logarithmic standard deviation (log[SD], i.e., treatment response heterogeneity) in verum (i.e., treatment) and placebo groups were included. Meta-regression analyses were performed with respect to variability of body weight after treatment and potential predictors. RESULTS A total of 120 RCTs with a total of 43 663 participants were analysed. A slightly larger treatment response heterogeneity was shown in the verum groups, with a median log(SD) of 2.83 compared to 2.79 from placebo. After full adjustment in the meta-regression model, the difference in body weight log(SD) was -0.026 (95% confidence interval -0.044; 0.008), with greater variability in the placebo groups. Scatterplots did not show any slope divergence (i.e., interaction) between clinical predictors and the respective treatment (verum or placebo). CONCLUSIONS We found no major treatment response heterogeneity in RCTs of glucose-lowering drugs for body weight reduction in type 2 diabetes. The precision medicine approach may thus be of limited value in this setting.
Collapse
Affiliation(s)
- Kris G Vargas
- Division of Cardiology, Pulmonology and Vascular Medicine, Department of Internal Medicine, Medical Faculty and University Hospital Düsseldorf, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
- Nuffield Department of Population Health, University of Oxford, Oxford, UK
| | - Tobias Rütten
- Institute for Biometrics and Epidemiology, German Diabetes Center, Leibniz Center for Diabetes Research at Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Benedikt Siemes
- Institute for Biometrics and Epidemiology, German Diabetes Center, Leibniz Center for Diabetes Research at Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Maximilian Brockmeyer
- Division of Cardiology, Pulmonology and Vascular Medicine, Department of Internal Medicine, Medical Faculty and University Hospital Düsseldorf, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Claudio Parco
- Division of Cardiology, Pulmonology and Vascular Medicine, Department of Internal Medicine, Medical Faculty and University Hospital Düsseldorf, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Alexander Hoss
- Division of Cardiology, Pulmonology and Vascular Medicine, Department of Internal Medicine, Medical Faculty and University Hospital Düsseldorf, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Sabrina Schlesinger
- Institute for Biometrics and Epidemiology, German Diabetes Center, Leibniz Center for Diabetes Research at Heinrich Heine University Düsseldorf, Düsseldorf, Germany
- German Center for Diabetes Research, Partner Düsseldorf, München-Neuherberg, Germany
| | - Christian Jung
- Division of Cardiology, Pulmonology and Vascular Medicine, Department of Internal Medicine, Medical Faculty and University Hospital Düsseldorf, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Michael Roden
- German Center for Diabetes Research, Partner Düsseldorf, München-Neuherberg, Germany
- Department of Endocrinology and Diabetology, Medical Faculty and University Hospital Düsseldorf, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
- Institute for Clinical Diabetology, German Diabetes Center, Leibniz Center for Diabetes Research at Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Malte Kelm
- Division of Cardiology, Pulmonology and Vascular Medicine, Department of Internal Medicine, Medical Faculty and University Hospital Düsseldorf, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
- Cardiovascular Research Institute Düsseldorf (CARID), Medical Faculty and University Hospital Düsseldorf, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Georg Wolff
- Division of Cardiology, Pulmonology and Vascular Medicine, Department of Internal Medicine, Medical Faculty and University Hospital Düsseldorf, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Oliver Kuss
- Institute for Biometrics and Epidemiology, German Diabetes Center, Leibniz Center for Diabetes Research at Heinrich Heine University Düsseldorf, Düsseldorf, Germany
- German Center for Diabetes Research, Partner Düsseldorf, München-Neuherberg, Germany
- Centre for Health and Society, Faculty of Medicine, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| |
Collapse
|
5
|
Andaur Navarro CL, Damen JAA, Ghannad M, Dhiman P, van Smeden M, Reitsma JB, Collins GS, Riley RD, Moons KGM, Hooft L. SPIN-PM: a consensus framework to evaluate the presence of spin in studies on prediction models. J Clin Epidemiol 2024; 170:111364. [PMID: 38631529 DOI: 10.1016/j.jclinepi.2024.111364] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Revised: 04/01/2024] [Accepted: 04/08/2024] [Indexed: 04/19/2024]
Abstract
OBJECTIVES To develop a framework to identify and evaluate spin practices and its facilitators in studies on clinical prediction model regardless of the modeling technique. STUDY DESIGN AND SETTING We followed a three-phase consensus process: (1) premeeting literature review to generate items to be included; (2) a series of structured meetings to provide comments discussed and exchanged viewpoints on items to be included with a panel of experienced researchers; and (3) postmeeting review on final list of items and examples to be included. Through this iterative consensus process, a framework was derived after all panel's researchers agreed. RESULTS This consensus process involved a panel of eight researchers and resulted in SPIN-Prediction Models which consists of two categories of spin (misleading interpretation and misleading transportability), and within these categories, two forms of spin (spin practices and facilitators of spin). We provide criteria and examples. CONCLUSION We proposed this guidance aiming to facilitate not only the accurate reporting but also an accurate interpretation and extrapolation of clinical prediction models which will likely improve the reporting quality of subsequent research, as well as reduce research waste.
Collapse
Affiliation(s)
- Constanza L Andaur Navarro
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands; Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.
| | - Johanna A A Damen
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands; Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Mona Ghannad
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands; Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Paula Dhiman
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK; NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Maarten van Smeden
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Johannes B Reitsma
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK; NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Richard D Riley
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham B15 2TT, UK
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands; Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Lotty Hooft
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands; Cochrane Netherlands, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
6
|
Siafarikas N. Personalized medicine in old age psychiatry and Alzheimer's disease. Front Psychiatry 2024; 15:1297798. [PMID: 38751423 PMCID: PMC11094449 DOI: 10.3389/fpsyt.2024.1297798] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/21/2023] [Accepted: 04/15/2024] [Indexed: 05/18/2024] Open
Abstract
Elderly patients show us unfolded lives with unique individual characteristics. An increasing life span is associated with increasing physical and mental disease burden. Alzheimer's disease (AD) is an increasing challenge in old age. AD cannot be cured but it can be treated. The complexity of old age and AD offer targets for personalized medicine (PM). Targets for stratification of patients, detection of patients at risk for AD or for future targeted therapy are plentiful and can be found in several omic-levels.
Collapse
Affiliation(s)
- Nikias Siafarikas
- Department of Geriatric Psychiatry, Akershus University Hospital, Lørenskog, Norway
| |
Collapse
|
7
|
Maroufi SF, Doğruel Y, Pour-Rashidi A, Kohli GS, Parker CT, Uchida T, Asfour MZ, Martin C, Nizzola M, De Bonis A, Tawfik-Helika M, Tavallai A, Cohen-Gadol AA, Palmisciano P. Current status of artificial intelligence technologies in pituitary adenoma surgery: a scoping review. Pituitary 2024; 27:91-128. [PMID: 38183582 DOI: 10.1007/s11102-023-01369-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 11/27/2023] [Indexed: 01/08/2024]
Abstract
PURPOSE Pituitary adenoma surgery is a complex procedure due to critical adjacent neurovascular structures, variations in size and extensions of the lesions, and potential hormonal imbalances. The integration of artificial intelligence (AI) and machine learning (ML) has demonstrated considerable potential in assisting neurosurgeons in decision-making, optimizing surgical outcomes, and providing real-time feedback. This scoping review comprehensively summarizes the current status of AI/ML technologies in pituitary adenoma surgery, highlighting their strengths and limitations. METHODS PubMed, Embase, Web of Science, and Scopus were searched following the PRISMA-ScR guidelines. Studies discussing the use of AI/ML in pituitary adenoma surgery were included. Eligible studies were grouped to analyze the different outcomes of interest of current AI/ML technologies. RESULTS Among the 2438 identified articles, 44 studies met the inclusion criteria, with a total of seventeen different algorithms utilized across all studies. Studies were divided into two groups based on their input type: clinicopathological and imaging input. The four main outcome variables evaluated in the studies included: outcome (remission, recurrence or progression, gross-total resection, vision improvement, and hormonal recovery), complications (CSF leak, readmission, hyponatremia, and hypopituitarism), cost, and adenoma-related factors (aggressiveness, consistency, and Ki-67 labeling) prediction. Three studies focusing on workflow analysis and real-time navigation were discussed separately. CONCLUSION AI/ML modeling holds promise for improving pituitary adenoma surgery by enhancing preoperative planning and optimizing surgical strategies. However, addressing challenges such as algorithm selection, performance evaluation, data heterogeneity, and ethics is essential to establish robust and reliable ML models that can revolutionize neurosurgical practice and benefit patients.
Collapse
Affiliation(s)
- Seyed Farzad Maroufi
- Faculty of Medicine, Tehran University of Medical Sciences, Tehran, Iran
- Neurosurgical Research Network (NRN), Universal Scientific Education and Research Network (USERN), Tehran, Iran
| | - Yücel Doğruel
- Department of Neurosurgery, Yeditepe University School of Medicine, Istanbul, Turkey
| | - Ahmad Pour-Rashidi
- Department of Neurosurgery, Sina Hospital, Tehran University of Medical Sciences, Tehran, Iran
| | - Gurkirat S Kohli
- Department of Neurosurgery, University of Rochester School of Medicine and Dentistry, Rochester, NY, USA
| | | | - Tatsuya Uchida
- Department of Neurosurgery, Stanford University, Palo Alto, CA, USA
| | - Mohamed Z Asfour
- Department of Neurosurgery, Nasser Institute for Research and Treatment Hospital, Cairo, Egypt
| | - Clara Martin
- Department of Neurosurgery, Hospital de Alta Complejidad en Red "El Cruce", Florencio Varela, Buenos Aires, Argentina
| | | | - Alessandro De Bonis
- Department of Neurosurgery and Gamma Knife Radiosurgery, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | | | - Amin Tavallai
- Department of Pediatric Neurosurgery, Children's Medical Center Hospital, Tehran University of Medical Sciences, Tehran, Iran
| | | | - Paolo Palmisciano
- Department of Neurological Surgery, University of California, Davis, 4860 Y Street, Suite 3740, Sacramento, CA, 95817, USA.
| |
Collapse
|
8
|
Ktena I, Wiles O, Albuquerque I, Rebuffi SA, Tanno R, Roy AG, Azizi S, Belgrave D, Kohli P, Cemgil T, Karthikesalingam A, Gowal S. Generative models improve fairness of medical classifiers under distribution shifts. Nat Med 2024; 30:1166-1173. [PMID: 38600282 PMCID: PMC11031395 DOI: 10.1038/s41591-024-02838-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2023] [Accepted: 01/26/2024] [Indexed: 04/12/2024]
Abstract
Domain generalization is a ubiquitous challenge for machine learning in healthcare. Model performance in real-world conditions might be lower than expected because of discrepancies between the data encountered during deployment and development. Underrepresentation of some groups or conditions during model development is a common cause of this phenomenon. This challenge is often not readily addressed by targeted data acquisition and 'labeling' by expert clinicians, which can be prohibitively expensive or practically impossible because of the rarity of conditions or the available clinical expertise. We hypothesize that advances in generative artificial intelligence can help mitigate this unmet need in a steerable fashion, enriching our training dataset with synthetic examples that address shortfalls of underrepresented conditions or subgroups. We show that diffusion models can automatically learn realistic augmentations from data in a label-efficient manner. We demonstrate that learned augmentations make models more robust and statistically fair in-distribution and out of distribution. To evaluate the generality of our approach, we studied three distinct medical imaging contexts of varying difficulty: (1) histopathology, (2) chest X-ray and (3) dermatology images. Complementing real samples with synthetic ones improved the robustness of models in all three medical tasks and increased fairness by improving the accuracy of clinical diagnosis within underrepresented groups, especially out of distribution.
Collapse
|
9
|
Brooks JM, Chapman CG, Chen BK, Floyd SB, Hikmet N. Assessing the properties of patient-specific treatment effect estimates from causal forest algorithms under essential heterogeneity. BMC Med Res Methodol 2024; 24:66. [PMID: 38481139 PMCID: PMC10935905 DOI: 10.1186/s12874-024-02187-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Accepted: 02/21/2024] [Indexed: 03/17/2024] Open
Abstract
BACKGROUND Treatment variation from observational data has been used to estimate patient-specific treatment effects. Causal Forest Algorithms (CFAs) developed for this task have unknown properties when treatment effect heterogeneity from unmeasured patient factors influences treatment choice - essential heterogeneity. METHODS We simulated eleven populations with identical treatment effect distributions based on patient factors. The populations varied in the extent that treatment effect heterogeneity influenced treatment choice. We used the generalized random forest application (CFA-GRF) to estimate patient-specific treatment effects for each population. Average differences between true and estimated effects for patient subsets were evaluated. RESULTS CFA-GRF performed well across the population when treatment effect heterogeneity did not influence treatment choice. Under essential heterogeneity, however, CFA-GRF yielded treatment effect estimates that reflected true treatment effects only for treated patients and were on average greater than true treatment effects for untreated patients. CONCLUSIONS Patient-specific estimates produced by CFAs are sensitive to why patients in real-world practice make different treatment choices. Researchers using CFAs should develop conceptual frameworks of treatment choice prior to estimation to guide estimate interpretation ex post.
Collapse
Affiliation(s)
- John M Brooks
- Center for Effectiveness Research in Orthopaedics - Arnold School of Public Health Greenville, 915 Greene Street #302D, Columbia, SC, 29208-0001, USA.
- University of South Carolina Arnold School of Public Health, Health Services Policy & Management, Columbia, SC, USA.
| | - Cole G Chapman
- Department of Pharmacy Practice and Science Iowa City, University of Iowa, Iowa, USA
- Center for Effectiveness Research in Orthopaedics, Greenville, SC, USA
| | - Brian K Chen
- University of South Carolina Arnold School of Public Health, Health Services Policy & Management, Columbia, SC, USA
- Center for Effectiveness Research in Orthopaedics, Greenville, SC, USA
| | - Sarah B Floyd
- Center for Effectiveness Research in Orthopaedics, Greenville, SC, USA
- Clemson University College of Behavioral Social and Health Sciences, Public Health Sciences, Clemson, South Carolina, USA
| | - Neset Hikmet
- Center for Effectiveness Research in Orthopaedics, Greenville, SC, USA
- Department of Integrated Information Technology, Innovation Think Tank Lab @ USC, University of South Carolina College of Engineering and Computing, Columbia, SC, USA
| |
Collapse
|
10
|
Meeus M, Beirnaert C, Mahieu L, Laukens K, Meysman P, Mulder A, Van Laere D. Clinical Decision Support for Improved Neonatal Care: The Development of a Machine Learning Model for the Prediction of Late-onset Sepsis and Necrotizing Enterocolitis. J Pediatr 2024; 266:113869. [PMID: 38065281 DOI: 10.1016/j.jpeds.2023.113869] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/20/2023] [Revised: 11/24/2023] [Accepted: 12/04/2023] [Indexed: 01/08/2024]
Abstract
OBJECTIVE To develop an artificial intelligence-based software system for predicting late-onset sepsis (LOS) and necrotizing enterocolitis (NEC) in infants admitted to the neonatal intensive care unit (NICU). STUDY DESIGN Single-center, retrospective cohort study, conducted in the NICU of the Antwerp University Hospital. Continuous monitoring data of 865 preterm infants born at <32 weeks gestational age, admitted to the NICU in the first week of life, were used to train an XGBoost machine learning (ML) algorithm for LOS and NEC prediction in a cross-validated setup. Afterward, the model's performance was assessed on an independent test set of 148 patients (internal validation). RESULTS The ML model delivered hourly risk predictions with an overall sensitivity of 69% (142/206) for all LOS/NEC episodes and 81% (67/83) for severe LOS/NEC episodes. The model showed a median time gain of ≤10 hours (IQR, 3.1-21.0 hours), compared with historical clinical diagnosis. On the complete retrospective dataset, the ML model made 721 069 predictions, of which 9805 (1.3%) depicted a LOS/NEC probability of ≥0.15, resulting in a total alarm rate of <1 patient alarm-day per week. The model reached a similar performance on the internal validation set. CONCLUSIONS Artificial intelligence technology can assist clinicians in the early detection of LOS and NEC in the NICU, which potentially can result in clinical and socioeconomic benefits. Additional studies are required to quantify further the effect of combining artificial and human intelligence on patient outcomes in the NICU.
Collapse
Affiliation(s)
- Marisse Meeus
- Department of Neonatal Intensive Care, Antwerp University Hospital, Edegem, Belgium; Laboratory of Experimental Medicine and Pediatrics, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerpen, Belgium.
| | - Charlie Beirnaert
- Department of Neonatal Intensive Care, Antwerp University Hospital, Edegem, Belgium; Innocens BV, Antwerpen, Belgium; Department of Computer Science, University of Antwerp, Antwerpen, Belgium
| | - Ludo Mahieu
- Department of Neonatal Intensive Care, Antwerp University Hospital, Edegem, Belgium; Laboratory of Experimental Medicine and Pediatrics, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerpen, Belgium
| | - Kris Laukens
- Department of Computer Science, University of Antwerp, Antwerpen, Belgium
| | - Pieter Meysman
- Department of Computer Science, University of Antwerp, Antwerpen, Belgium
| | - Antonius Mulder
- Department of Neonatal Intensive Care, Antwerp University Hospital, Edegem, Belgium; Laboratory of Experimental Medicine and Pediatrics, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerpen, Belgium
| | - David Van Laere
- Department of Neonatal Intensive Care, Antwerp University Hospital, Edegem, Belgium; Laboratory of Experimental Medicine and Pediatrics, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerpen, Belgium; Innocens BV, Antwerpen, Belgium
| |
Collapse
|
11
|
Benzaquen J, Hofman P, Lopez S, Leroy S, Rouis N, Padovani B, Fontas E, Marquette CH, Boutros J. Integrating artificial intelligence into lung cancer screening: a randomised controlled trial protocol. BMJ Open 2024; 14:e074680. [PMID: 38355174 PMCID: PMC10868245 DOI: 10.1136/bmjopen-2023-074680] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/13/2023] [Accepted: 12/21/2023] [Indexed: 02/16/2024] Open
Abstract
INTRODUCTION Lung cancer (LC) is the most common cause of cancer-related deaths worldwide. Its early detection can be achieved with a CT scan. Two large randomised trials proved the efficacy of low-dose CT (LDCT)-based lung cancer screening (LCS) in high-risk populations. The decrease in specific mortality is 20%-25%.Nonetheless, implementing LCS on a large scale faces obstacles due to the low number of thoracic radiologists and CT scans available for the eligible population and the high frequency of false-positive screening results and the long period of indeterminacy of nodules that can reach up to 24 months, which is a source of prolonged anxiety and multiple costly examinations with possible side effects.Deep learning, an artificial intelligence solution has shown promising results in retrospective trials detecting lung nodules and characterising them. However, until now no prospective studies have demonstrated their importance in a real-life setting. METHODS AND ANALYSIS This open-label randomised controlled study focuses on LCS for patients aged 50-80 years, who smoked more than 20 pack-years, whether active or quit smoking less than 15 years ago. Its objective is to determine whether assisting a multidisciplinary team (MDT) with a 3D convolutional network-based analysis of screening chest CT scans accelerates the definitive classification of nodules into malignant or benign. 2722 patients will be included with the aim to demonstrate a 3-month reduction in the delay between lung nodule detection and its definitive classification into benign or malignant. ETHICS AND DISSEMINATION The sponsor of this study is the University Hospital of Nice. The study was approved for France by the ethical committee CPP (Comités de Protection des Personnes) Sud-Ouest et outre-mer III (No. 2022-A01543-40) and the Agence Nationale du Medicament et des produits de Santé (Ministry of Health) in December 2023. The findings of the trial will be disseminated through peer-reviewed journals and national and international conference presentations. TRIAL REGISTRATION NUMBER NCT05704920.
Collapse
Affiliation(s)
- Jonathan Benzaquen
- Department of Pulmonary Medicine and Thoracic Oncology, FHU OncoAge, IHU RespirERA, Centre Hospitalier Universitaire de Nice, Nice, France
| | - Paul Hofman
- Laboratory of Clinical and Experimental Pathology, FHU OncoAge, IHU RespirERA, Universite Cote d'Azur, Centre hospitalier Universitaire de Nice, Nice, France
| | | | - Sylvie Leroy
- Department of Pulmonary Medicine and Thoracic Oncology, FHU OncoAge, IHU RespirERA, Centre Hospitalier Universitaire de Nice, Nice, France
- Institut de Pharmacologie Moléculaire et Cellulaire, Nice, France
| | - Nesrine Rouis
- Department of Pulmonary Medicine and Thoracic Oncology, FHU OncoAge, IHU RespirERA, Centre Hospitalier Universitaire de Nice, Nice, France
| | - Bernard Padovani
- Department of Radiology, Centre Hospitalier Universitaire de Nice, Nice, France
| | - Eric Fontas
- Délégation à la Recherche Clinique et à l'Innovation, Centre Hospitalier Universitaire de Nice, Nice, France
| | - Charles Hugo Marquette
- Department of Pulmonary Medicine and Thoracic Oncology, FHU OncoAge, IHU RespirERA, Centre Hospitalier Universitaire de Nice, Nice, France
| | - Jacques Boutros
- Department of Pulmonary Medicine and Thoracic Oncology, FHU OncoAge, IHU RespirERA, Centre Hospitalier Universitaire de Nice, Nice, France
| |
Collapse
|
12
|
Chari A, Adler S, Wagstyl K, Seunarine K, Tahir MZ, Moeller F, Thornton R, Boyd S, Das K, Cooray G, Smith S, D'Arco F, Baldeweg T, Eltze C, Cross JH, Tisdall MM. Lesion detection in epilepsy surgery: Lessons from a prospective evaluation of a machine learning algorithm. Dev Med Child Neurol 2024; 66:216-225. [PMID: 37559345 DOI: 10.1111/dmcn.15727] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Revised: 06/11/2023] [Accepted: 07/03/2023] [Indexed: 08/11/2023]
Abstract
AIM To evaluate a lesion detection algorithm designed to detect focal cortical dysplasia (FCD) in children undergoing stereoelectroencephalography (SEEG) as part of their presurgical evaluation for drug-resistant epilepsy. METHOD This was a prospective, single-arm, interventional study (Idea, Development, Exploration, Assessment, and Long-Term Follow-Up phase 1/2a). After routine SEEG planning, structural magnetic resonance imaging sequences were run through an FCD lesion detection algorithm to identify putative clusters. If the top three clusters were not already sampled, up to three additional SEEG electrodes were added. The primary outcome measure was the proportion of patients who had additional electrode contacts in the SEEG-defined seizure-onset zone (SOZ). RESULTS Twenty patients (median age 12 years, range 4-18 years) were enrolled, one of whom did not undergo SEEG. Additional electrode contacts were part of the SOZ in 1 out of 19 patients while 3 out of 19 patients had clusters that were part of the SOZ but they were already implanted. A total of 16 additional electrodes were implanted in nine patients and there were no adverse events from the additional electrodes. INTERPRETATION We demonstrate early-stage prospective clinical validation of a machine learning lesion detection algorithm used to aid the identification of the SOZ in children undergoing SEEG. We share key lessons learnt from this evaluation and emphasize the importance of robust prospective evaluation before routine clinical adoption of such algorithms. WHAT THIS PAPER ADDS The focal cortical dysplasia detection algorithm collocated with the seizure-onset zone (SOZ) in 4 out of 19 patients. The algorithm changed the resection boundaries in 1 of 19 patients undergoing stereoelectroencephalography for drug-resistant epilepsy. The patient with an altered resection due to the algorithm was seizure-free 1 year after resective surgery. Overall, the algorithm did not increase the proportion of patients in whom SOZ was identified.
Collapse
Affiliation(s)
- Aswin Chari
- Department of Neurosurgery, Great Ormond Street Hospital, London, UK
- Developmental Neuroscience, Institute of Child Health, University College London, London, UK
| | - Sophie Adler
- Developmental Neuroscience, Institute of Child Health, University College London, London, UK
| | - Konrad Wagstyl
- Developmental Neuroscience, Institute of Child Health, University College London, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Kiran Seunarine
- Developmental Neuroscience, Institute of Child Health, University College London, London, UK
| | - M Zubair Tahir
- Department of Neurosurgery, Great Ormond Street Hospital, London, UK
| | | | - Rachel Thornton
- Department of Neurophysiology, Addenbrooke's Hospital, Cambridge, UK
| | - Steward Boyd
- Department of Neurophysiology, Great Ormond Street Hospital, London, UK
| | - Krishna Das
- Department of Neurophysiology, Great Ormond Street Hospital, London, UK
- Department of Neurology, Great Ormond Street Hospital, London, UK
| | - Gerald Cooray
- Department of Neurophysiology, Great Ormond Street Hospital, London, UK
| | - Stuart Smith
- Department of Neurophysiology, Great Ormond Street Hospital, London, UK
| | - Felice D'Arco
- Department of Neuroradiology, Great Ormond Street Hospital, London, UK
| | - Torsten Baldeweg
- Developmental Neuroscience, Institute of Child Health, University College London, London, UK
| | - Christin Eltze
- Department of Neurology, Great Ormond Street Hospital, London, UK
| | - J Helen Cross
- Developmental Neuroscience, Institute of Child Health, University College London, London, UK
- Department of Neurology, Great Ormond Street Hospital, London, UK
| | - Martin M Tisdall
- Department of Neurosurgery, Great Ormond Street Hospital, London, UK
- Developmental Neuroscience, Institute of Child Health, University College London, London, UK
| |
Collapse
|
13
|
Sen SK, Green ED, Hutter CM, Craven M, Ideker T, Di Francesco V. Opportunities for basic, clinical, and bioethics research at the intersection of machine learning and genomics. CELL GENOMICS 2024; 4:100466. [PMID: 38190108 PMCID: PMC10794834 DOI: 10.1016/j.xgen.2023.100466] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Revised: 07/14/2023] [Accepted: 11/20/2023] [Indexed: 01/09/2024]
Abstract
The data-intensive fields of genomics and machine learning (ML) are in an early stage of convergence. Genomics researchers increasingly seek to harness the power of ML methods to extract knowledge from their data; conversely, ML scientists recognize that genomics offers a wealth of large, complex, and well-annotated datasets that can be used as a substrate for developing biologically relevant algorithms and applications. The National Human Genome Research Institute (NHGRI) inquired with researchers working in these two fields to identify common challenges and receive recommendations to better support genomic research efforts using ML approaches. Those included increasing the amount and variety of training datasets by integrating genomic with multiomics, context-specific (e.g., by cell type), and social determinants of health datasets; reducing the inherent biases of training datasets; prioritizing transparency and interpretability of ML methods; and developing privacy-preserving technologies for research participants' data.
Collapse
Affiliation(s)
- Shurjo K Sen
- National Human Genome Research Institute, National Institutes of Health, Bethesda, MD 20892, USA.
| | - Eric D Green
- National Human Genome Research Institute, National Institutes of Health, Bethesda, MD 20892, USA
| | - Carolyn M Hutter
- National Human Genome Research Institute, National Institutes of Health, Bethesda, MD 20892, USA
| | - Mark Craven
- Department of Computer Sciences, University of Wisconsin-Madison, Madison, WI 53792, USA; Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, Madison, WI 53792, USA
| | - Trey Ideker
- Division of Genetics, Department of Medicine, University of California San Diego, La Jolla, CA 92093, USA
| | - Valentina Di Francesco
- National Human Genome Research Institute, National Institutes of Health, Bethesda, MD 20892, USA.
| |
Collapse
|
14
|
Lee KH, Choi GH, Yun J, Choi J, Goh MJ, Sinn DH, Jin YJ, Kim MA, Yu SJ, Jang S, Lee SK, Jang JW, Lee JS, Kim DY, Cho YY, Kim HJ, Kim S, Kim JH, Kim N, Kim KM. Machine learning-based clinical decision support system for treatment recommendation and overall survival prediction of hepatocellular carcinoma: a multi-center study. NPJ Digit Med 2024; 7:2. [PMID: 38182886 PMCID: PMC10770025 DOI: 10.1038/s41746-023-00976-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 11/29/2023] [Indexed: 01/07/2024] Open
Abstract
The treatment decisions for patients with hepatocellular carcinoma are determined by a wide range of factors, and there is a significant difference between the recommendations of widely used staging systems and the actual initial treatment choices. Herein, we propose a machine learning-based clinical decision support system suitable for use in multi-center settings. We collected data from nine institutions in South Korea for training and validation datasets. The internal and external datasets included 935 and 1750 patients, respectively. We developed a model with 20 clinical variables consisting of two stages: the first stage which recommends initial treatment using an ensemble voting machine, and the second stage, which predicts post-treatment survival using a random survival forest algorithm. We derived the first and second treatment options from the results with the highest and the second-highest probabilities given by the ensemble model and predicted their post-treatment survival. When only the first treatment option was accepted, the mean accuracy of treatment recommendation in the internal and external datasets was 67.27% and 55.34%, respectively. The accuracy increased to 87.27% and 86.06%, respectively, when the second option was included as the correct answer. Harrell's C index, integrated time-dependent AUC curve, and integrated Brier score of survival prediction in the internal and external datasets were 0.8381 and 0.7767, 91.89 and 86.48, 0.12, and 0.14, respectively. The proposed system can assist physicians by providing data-driven predictions for reference from other larger institutions or other physicians within the same institution when making treatment decisions.
Collapse
Affiliation(s)
- Kyung Hwa Lee
- Department of Radiation Oncology, Korea University Guro Hospital, Korea University College of Medicine, Seoul, Republic of Korea
| | - Gwang Hyeon Choi
- Department of Internal Medicine, Seoul National University Bundang Hospital, Seoul National University, Seongnam, Republic of Korea
| | - Jihye Yun
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Jonggi Choi
- Department of Gastroenterology, Asan Liver Center, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Myung Ji Goh
- Department of Internal Medicine, Samsung Medical Center, Seoul, Republic of Korea
| | - Dong Hyun Sinn
- Department of Internal Medicine, Samsung Medical Center, Seoul, Republic of Korea
| | - Young Joo Jin
- Department of Internal Medicine, Inha University Hospital, Incheon, Republic of Korea
| | - Minseok Albert Kim
- Department of Internal Medicine, Seoul National University Hospital, Seoul National University, Seoul, Republic of Korea
| | - Su Jong Yu
- Department of Internal Medicine, Seoul National University Hospital, Seoul National University, Seoul, Republic of Korea
| | - Sangmi Jang
- Department of Internal Medicine, Seoul National University Bundang Hospital, Seoul National University, Seongnam, Republic of Korea
- Department of Internal Medicine, Inha University Hospital, Incheon, Republic of Korea
| | - Soon Kyu Lee
- Department of Internal Medicine, Seoul St. Mary's Hospital, Seoul, Republic of Korea
- Department of Internal Medicine, Incheon St. Mary's Hospital, Incheon, Republic of Korea
| | - Jeong Won Jang
- Department of Internal Medicine, Seoul St. Mary's Hospital, Seoul, Republic of Korea
| | - Jae Seung Lee
- Department of Internal Medicine, Seoul Severance Hospital, Seoul, Republic of Korea
| | - Do Young Kim
- Department of Internal Medicine, Seoul Severance Hospital, Seoul, Republic of Korea
| | - Young Youn Cho
- Department of Internal Medicine, Chung-Ang University Hospital, Seoul, Republic of Korea
| | - Hyung Joon Kim
- Department of Internal Medicine, Chung-Ang University Hospital, Seoul, Republic of Korea
| | - Sehwa Kim
- Department of Internal Medicine, Korea University Guro Hospital, Korea University College of Medicine, Seoul, Republic of Korea
- Department of Internal Medicine, Bundang Jesaeng General Hospital, Seongnam, Republic of Korea
| | - Ji Hoon Kim
- Department of Internal Medicine, Korea University Guro Hospital, Korea University College of Medicine, Seoul, Republic of Korea
| | - Namkug Kim
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea.
- Department of Convergence Medicine, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea.
| | - Kang Mo Kim
- Department of Gastroenterology, Asan Liver Center, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea.
| |
Collapse
|
15
|
Pfob A, Hillen C, Seitz K, Griewing S, Becker S, Bayer C, Wagner U, Fasching P, Wallwiener M. Status quo and future directions of digitalization in gynecology and obstetrics in Germany: a survey of the commission Digital Medicine of the German Society for Gynecology and Obstetrics. Arch Gynecol Obstet 2024; 309:195-204. [PMID: 37755531 PMCID: PMC10769997 DOI: 10.1007/s00404-023-07222-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Accepted: 09/05/2023] [Indexed: 09/28/2023]
Abstract
PURPOSE Digitalization plays a critical role and is beginning to impact every part of the patient journey, from drug discovery and data collection to treatment and patient-reported outcomes. We aimed to evaluate the status quo and future directions of digital medicine in the specialty of gynecology and obstetrics in Germany. METHODS An anonymous questionnaire was distributed via the German Society of Gynecology and Obstetrics newsletter in December 2022. The questionnaire covered the domains baseline demographic information, telemedicine, digital health applications (DIGAs), and future expectations. RESULTS In all, 91 participants completed the survey. Median age was 34 years; 67.4% (60 of 89) were female and 32.6% (29 of 89) were male. About 10% (9 of 88) have prescribed DIGAs to date and 14% (12 of 86) offer telemedical appointments. Among those who do not use digital medicine, very few plan to do so in the near future. Reasons include missing software interfaces, lack of time to try out new things, lack of knowledge, lack of monetary compensation (66.3%), and employee concerns. A majority agreed that digitalization will help to save time and improve patient care and that intelligent algorithms will aid clinicians in providing patient care to women. CONCLUSIONS The status quo and future directions of digital medicine in gynecology and obstetrics in Germany are characterized by contradicting expectations regarding the benefits of digital medicine and its actual implementation in clinical routine. This represents an important call to action to meet the requirements of modern patient care.
Collapse
Affiliation(s)
- André Pfob
- Department of Obstetrics & Gynecology, Heidelberg University Hospital, Heidelberg, Germany.
- National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany.
| | - Christoph Hillen
- Department of Gynecology and Gynecologic Oncology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Katharina Seitz
- University Hospital Erlangen, Comprehensive Cancer Center Erlangen-EMN, Department of Gynecology and Obstetrics, Friedrich-Alexander University Erlangen-Nuremberg, Erlangen, Germany
| | - Sebastian Griewing
- Department of Gynecology and Obstetrics, University Hospital Marburg, Philipps-University Marburg, Baldingerstraße, 35043, Marburg, Germany
| | - Sven Becker
- Department of Gynecology, University Hospital, Frankfurt am Main, Germany
| | | | - Uwe Wagner
- Department of Gynecology and Obstetrics, University Hospital Marburg, Philipps-University Marburg, Baldingerstraße, 35043, Marburg, Germany
| | - Peter Fasching
- University Hospital Erlangen, Comprehensive Cancer Center Erlangen-EMN, Department of Gynecology and Obstetrics, Friedrich-Alexander University Erlangen-Nuremberg, Erlangen, Germany
| | - Markus Wallwiener
- Department of Obstetrics & Gynecology, Heidelberg University Hospital, Heidelberg, Germany
| |
Collapse
|
16
|
Buchman DZ, Imahori D, Lo C, Hui K, Walker C, Shaw J, Davis KD. The Influence of Using Novel Predictive Technologies on Judgments of Stigma, Empathy, and Compassion among Healthcare Professionals. AJOB Neurosci 2024; 15:32-45. [PMID: 37450417 DOI: 10.1080/21507740.2023.2225470] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/18/2023]
Abstract
BACKGROUND Our objective was to evaluate whether the description of a machine learning (ML) app or brain imaging technology to predict the onset of schizophrenia or alcohol use disorder (AUD) influences healthcare professionals' judgments of stigma, empathy, and compassion. METHODS We randomized healthcare professionals (N = 310) to one vignette about a person whose clinician seeks to predict schizophrenia or an AUD, using a ML app, brain imaging, or a psychosocial assessment. Participants used scales to measure their judgments of stigma, empathy, and compassion. RESULTS Participants randomized to the ML vignette endorsed less anger and more fear relative to the psychosocial vignette, and the brain imaging vignette elicited higher pity ratings. The brain imaging and ML vignettes evoked lower personal responsibility judgments compared to the psychosocial vignette. Physicians and nurses reported less empathy than clinical psychologists. CONCLUSIONS The use of predictive technologies may reinforce essentialist views about mental health and substance use that may increase specific aspects of stigma and reduce others.
Collapse
Affiliation(s)
- Daniel Z Buchman
- Centre for Addiction and Mental Health
- Dalla Lana School of Public Health, University of Toronto
- University of Toronto Joint Centre for Bioethics
| | | | - Christopher Lo
- Dalla Lana School of Public Health, University of Toronto
- Temerty Faculty of Medicine, University of Toronto
- College of Healthcare Sciences, James Cook University, Singapore
| | - Katrina Hui
- Centre for Addiction and Mental Health
- Temerty Faculty of Medicine, University of Toronto
| | | | - James Shaw
- University of Toronto Joint Centre for Bioethics
- Temerty Faculty of Medicine, University of Toronto
| | - Karen D Davis
- Temerty Faculty of Medicine, University of Toronto
- Krembil Brain Institute, University Health Network
| |
Collapse
|
17
|
Kerr WT, McFarlane KN. Machine Learning and Artificial Intelligence Applications to Epilepsy: a Review for the Practicing Epileptologist. Curr Neurol Neurosci Rep 2023; 23:869-879. [PMID: 38060133 DOI: 10.1007/s11910-023-01318-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/24/2023] [Indexed: 12/08/2023]
Abstract
PURPOSE OF REVIEW Machine Learning (ML) and Artificial Intelligence (AI) are data-driven techniques to translate raw data into applicable and interpretable insights that can assist in clinical decision making. Some of these tools have extremely promising initial results, earning both great excitement and creating hype. This non-technical article reviews recent developments in ML/AI in epilepsy to assist the current practicing epileptologist in understanding both the benefits and limitations of integrating ML/AI tools into their clinical practice. RECENT FINDINGS ML/AI tools have been developed to assist clinicians in almost every clinical decision including (1) predicting future epilepsy in people at risk, (2) detecting and monitoring for seizures, (3) differentiating epilepsy from mimics, (4) using data to improve neuroanatomic localization and lateralization, and (5) tracking and predicting response to medical and surgical treatments. We also discuss practical, ethical, and equity considerations in the development and application of ML/AI tools including chatbots based on Large Language Models (e.g., ChatGPT). ML/AI tools will change how clinical medicine is practiced, but, with rare exceptions, the transferability to other centers, effectiveness, and safety of these approaches have not yet been established rigorously. In the future, ML/AI will not replace epileptologists, but epileptologists with ML/AI will replace epileptologists without ML/AI.
Collapse
Affiliation(s)
- Wesley T Kerr
- Department of Neurology, University of Pittsburgh, 3471 Fifth Ave, Kaufmann 811.22, Pittsburgh, PA, 15213, USA.
- Department of Biomedical Informatics, University of Pittsburgh, 3471 Fifth Ave, Kaufmann 811.22, Pittsburgh, PA, 15213, USA.
- Department of Neurology, Michigan Medicine, University of Michigan, Ann Arbor, MI, USA.
| | - Katherine N McFarlane
- Department of Neurology, University of Pittsburgh, 3471 Fifth Ave, Kaufmann 811.22, Pittsburgh, PA, 15213, USA
| |
Collapse
|
18
|
Pham TD, Sun X. Wavelet scattering networks in deep learning for discovering protein markers in a cohort of Swedish rectal cancer patients. Cancer Med 2023; 12:21502-21518. [PMID: 38014709 PMCID: PMC10726782 DOI: 10.1002/cam4.6672] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Revised: 09/25/2023] [Accepted: 10/20/2023] [Indexed: 11/29/2023] Open
Abstract
BACKGROUND Cancer biomarkers play a pivotal role in the diagnosis, prognosis, and treatment response prediction of the disease. In this study, we analyzed the expression levels of RhoB and DNp73 proteins in rectal cancer, as captured in immunohistochemical images, to predict the 5-year survival time of two patient groups: one with preoperative radiotherapy and one without. METHODS The utilization of deep convolutional neural networks in medical research, particularly in clinical cancer studies, has been gaining substantial attention. This success primarily stems from their ability to extract intricate image features that prove invaluable in machine learning. Another innovative method for extracting features at multiple levels is the wavelet-scattering network. Our study combines the strengths of these two convolution-based approaches to robustly extract image features related to protein expression. RESULTS The efficacy of our approach was evaluated across various tissue types, including tumor, biopsy, metastasis, and adjacent normal tissue. Statistical assessments demonstrated exceptional performance across a range of metrics, including prediction accuracy, classification accuracy, precision, and the area under the receiver operating characteristic curve. CONCLUSION These results underscore the potential of dual convolutional learning to assist clinical researchers in the timely validation and discovery of cancer biomarkers.
Collapse
Affiliation(s)
- Tuan D. Pham
- Barts and The London School of Medicine and Dentistry Queen MaryUniversity of London Turner StreetLondonUK
| | - Xiao‐Feng Sun
- Division of Oncology Department of Biomedical and Clinical SciencesLinkoping UniversityLinkopingSweden
| |
Collapse
|
19
|
Ledger A, Ceusters J, Valentin L, Testa A, Van Holsbeke C, Franchi D, Bourne T, Froyman W, Timmerman D, Van Calster B. Multiclass risk models for ovarian malignancy: an illustration of prediction uncertainty due to the choice of algorithm. BMC Med Res Methodol 2023; 23:276. [PMID: 38001421 PMCID: PMC10668424 DOI: 10.1186/s12874-023-02103-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Accepted: 11/14/2023] [Indexed: 11/26/2023] Open
Abstract
BACKGROUND Assessing malignancy risk is important to choose appropriate management of ovarian tumors. We compared six algorithms to estimate the probabilities that an ovarian tumor is benign, borderline malignant, stage I primary invasive, stage II-IV primary invasive, or secondary metastatic. METHODS This retrospective cohort study used 5909 patients recruited from 1999 to 2012 for model development, and 3199 patients recruited from 2012 to 2015 for model validation. Patients were recruited at oncology referral or general centers and underwent an ultrasound examination and surgery ≤ 120 days later. We developed models using standard multinomial logistic regression (MLR), Ridge MLR, random forest (RF), XGBoost, neural networks (NN), and support vector machines (SVM). We used nine clinical and ultrasound predictors but developed models with or without CA125. RESULTS Most tumors were benign (3980 in development and 1688 in validation data), secondary metastatic tumors were least common (246 and 172). The c-statistic (AUROC) to discriminate benign from any type of malignant tumor ranged from 0.89 to 0.92 for models with CA125, from 0.89 to 0.91 for models without. The multiclass c-statistic ranged from 0.41 (SVM) to 0.55 (XGBoost) for models with CA125, and from 0.42 (SVM) to 0.51 (standard MLR) for models without. Multiclass calibration was best for RF and XGBoost. Estimated probabilities for a benign tumor in the same patient often differed by more than 0.2 (20% points) depending on the model. Net Benefit for diagnosing malignancy was similar for algorithms at the commonly used 10% risk threshold, but was slightly higher for RF at higher thresholds. Comparing models, between 3% (XGBoost vs. NN, with CA125) and 30% (NN vs. SVM, without CA125) of patients fell on opposite sides of the 10% threshold. CONCLUSION Although several models had similarly good performance, individual probability estimates varied substantially.
Collapse
Affiliation(s)
- Ashleigh Ledger
- Department of Development and Regeneration, KU Leuven, Herestraat 49 box 805, Leuven, 3000, Belgium
| | - Jolien Ceusters
- Department of Development and Regeneration, KU Leuven, Herestraat 49 box 805, Leuven, 3000, Belgium
- Department of Oncology, Leuven Cancer Institute, Laboratory of Tumor Immunology and Immunotherapy, KU Leuven, Leuven, Belgium
| | - Lil Valentin
- Department of Obstetrics and Gynecology, Skåne University Hospital, Malmö, Sweden
- Department of Clinical Sciences Malmö, Lund University, Malmö, Sweden
| | - Antonia Testa
- Department of Woman, Child and Public Health, Fondazione Policlinico Universitario A. Gemelli IRCCS, Rome, Italy
- Dipartimento Universitario Scienze della Vita e Sanità Pubblica, Università Cattolica del Sacro Cuore, Rome, Italy
| | | | - Dorella Franchi
- Preventive Gynecology Unit, Division of Gynecology, European Institute of Oncology IRCCS, Milan, Italy
| | - Tom Bourne
- Department of Development and Regeneration, KU Leuven, Herestraat 49 box 805, Leuven, 3000, Belgium
- Department of Obstetrics and Gynecology, University Hospitals Leuven, Leuven, Belgium
- Queen Charlotte's and Chelsea Hospital, Imperial College, London, UK
| | - Wouter Froyman
- Department of Development and Regeneration, KU Leuven, Herestraat 49 box 805, Leuven, 3000, Belgium
- Department of Obstetrics and Gynecology, University Hospitals Leuven, Leuven, Belgium
| | - Dirk Timmerman
- Department of Development and Regeneration, KU Leuven, Herestraat 49 box 805, Leuven, 3000, Belgium
- Department of Obstetrics and Gynecology, University Hospitals Leuven, Leuven, Belgium
| | - Ben Van Calster
- Department of Development and Regeneration, KU Leuven, Herestraat 49 box 805, Leuven, 3000, Belgium.
- Department of Biomedical Data Sciences, Leiden University Medical Centre (LUMC), Leiden, Netherlands.
- Leuven Unit for Health Technology Assessment Research (LUHTAR), KU Leuven, Leuven, Belgium.
| |
Collapse
|
20
|
Affiliation(s)
- Aaron E Carroll
- Center for Pediatric and Adolescent Comparative Effectiveness Research, Indiana University School of Medicine, Indianapolis
- Web and Social Media Editor, JAMA Pediatrics
| | - Dimitri A Christakis
- Center for Child Health, Behavior, and Development, Seattle Children's Research Institute, Seattle, Washington
- Editor, JAMA Pediatrics
| |
Collapse
|
21
|
Kuss O, Opitz ME, Brandstetter LV, Schlesinger S, Roden M, Hoyer A. How amenable is type 2 diabetes treatment for precision diabetology? A meta-regression of glycaemic control data from 174 randomised trials. Diabetologia 2023; 66:1622-1632. [PMID: 37338539 PMCID: PMC10390610 DOI: 10.1007/s00125-023-05951-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/06/2023] [Accepted: 05/16/2023] [Indexed: 06/21/2023]
Abstract
AIMS/HYPOTHESIS There are two prerequisites for the precision medicine approach to be beneficial for treated individuals. First, there must be treatment heterogeneity; second, in the case of treatment heterogeneity, we need to detect clinical predictors to identify people who would benefit from one treatment more than from others. There is an established meta-regression approach to assess these two prerequisites that relies on measuring the variability of a clinical outcome after treatment in placebo-controlled randomised trials. Our aim was to apply this approach to the treatment of type 2 diabetes. METHODS We performed a meta-regression analysis using information from 174 placebo-controlled randomised trials with 178 placebo and 272 verum (i.e. active treatment) arms including 86,940 participants with respect to the variability of glycaemic control as assessed by HbA1c after treatment and its potential predictors. RESULTS The adjusted difference in log(SD) values between the verum and placebo arms was 0.037 (95% CI: 0.004, 0.069). That is, we found a small increase in the variability of HbA1c values after treatment in the verum arms. In addition, one potentially relevant predictor for explaining this increase, drug class, was observed, and GLP-1 receptor agonists yielded the largest differences in log(SD) values. CONCLUSIONS/INTERPRETATION The potential of the precision medicine approach in the treatment of type 2 diabetes is modest at best, at least with regard to an improvement in glycaemic control. Our finding of a larger variability after treatment with GLP-1 receptor agonists in individuals with poor glycaemic control should be replicated and/or validated with other clinical outcomes and with different study designs. FUNDING The research reported here received no specific grant from any funding agency in the public, commercial or not-for-profit sectors. DATA AVAILABILITY Two datasets (one for the log[SD] and one for the baseline-corrected log[SD]) to reproduce the analyses from this paper are available on https://zenodo.org/record/7956635 .
Collapse
Affiliation(s)
- Oliver Kuss
- Institute for Biometrics and Epidemiology, German Diabetes Center, Leibniz Institute for Diabetes Research at Heinrich Heine University Düsseldorf, Düsseldorf, Germany.
- Centre for Health and Society, Faculty of Medicine, Heinrich Heine University Düsseldorf, Düsseldorf, Germany.
- German Center for Diabetes Research, Partner Düsseldorf, München-Neuherberg, Germany.
| | | | | | - Sabrina Schlesinger
- Institute for Biometrics and Epidemiology, German Diabetes Center, Leibniz Institute for Diabetes Research at Heinrich Heine University Düsseldorf, Düsseldorf, Germany
- German Center for Diabetes Research, Partner Düsseldorf, München-Neuherberg, Germany
| | - Michael Roden
- German Center for Diabetes Research, Partner Düsseldorf, München-Neuherberg, Germany
- Institute for Clinical Diabetology, German Diabetes Center, Leibniz Center for Diabetes Research at Heinrich Heine University Düsseldorf, Düsseldorf, Germany
- Department of Endocrinology and Diabetology, Medical Faculty and University Hospital Düsseldorf, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Annika Hoyer
- Biostatistics and Medical Biometry, Medical School EWL, Bielefeld University, Bielefeld, Germany
| |
Collapse
|
22
|
Sharma NK, Sarode SC. Artificial intelligence vs. evolving super-complex tumor intelligence: critical viewpoints. Front Artif Intell 2023; 6:1220744. [PMID: 37560445 PMCID: PMC10406576 DOI: 10.3389/frai.2023.1220744] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2023] [Accepted: 07/03/2023] [Indexed: 08/11/2023] Open
Abstract
Recent developments in various domains have led to a growing interest in the potential of artificial intelligence to enhance our lives and environments. In particular, the application of artificial intelligence in the management of complex human diseases, such as cancer, has garnered significant attention. The evolution of artificial intelligence is thought to be influenced by multiple factors, including human intervention and environmental factors. Similarly, tumors, being heterogeneous and complex diseases, continue to evolve due to changes in the physical, chemical, and biological environment. Additionally, the concept of cellular intelligence within biological systems has been recognized as a potential attribute of biological entities. Therefore, it is plausible that the tumor intelligence present in cancer cells of affected individuals could undergo super-evolution due to changes in the pro-tumor environment. Thus, a comparative analysis of the evolution of artificial intelligence and super-complex tumor intelligence could yield valuable insights to develop better artificial intelligence-based tools for cancer management.
Collapse
Affiliation(s)
- Nilesh Kumar Sharma
- Cancer and Translational Research Lab, Dr. D.Y. Patil Biotechnology & Bioinformatics Institute, Dr. D.Y. Patil Vidyapeeth, Pimpri, Pune, Maharashtra, India
| | - Sachin C. Sarode
- Department of Oral Pathology and Microbiology, Dr. D. Y. Patil Dental College and Hospital, Dr. D.Y. Patil Vidyapeeth, Pimpri, Pune, Maharashtra, India
| |
Collapse
|
23
|
Boussina A, Shashikumar S, Amrollahi F, Pour H, Hogarth M, Nemati S. Development & Deployment of a Real-time Healthcare Predictive Analytics Platform. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38083765 PMCID: PMC10802837 DOI: 10.1109/embc40787.2023.10340351] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
The deployment of predictive analytic algorithms that can safely and seamlessly integrate into existing healthcare workflows remains a significant challenge. Here, we present a scalable, cloud-based, fault-tolerant platform that is capable of extracting and processing electronic health record (EHR) data for any patient at any time following admission and transferring results back into the EHR. This platform has been successfully deployed within the UC San Diego Health system and utilizes interoperable data standards to enable portability.Clinical relevance- This platform is currently hosting a deep learning model for the early prediction of sepsis that is operational in two emergency departments.
Collapse
|
24
|
Li J, Cairns BJ, Li J, Zhu T. Generating synthetic mixed-type longitudinal electronic health records for artificial intelligent applications. NPJ Digit Med 2023; 6:98. [PMID: 37244963 DOI: 10.1038/s41746-023-00834-7] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2022] [Accepted: 05/05/2023] [Indexed: 05/29/2023] Open
Abstract
The recent availability of electronic health records (EHRs) have provided enormous opportunities to develop artificial intelligence (AI) algorithms. However, patient privacy has become a major concern that limits data sharing across hospital settings and subsequently hinders the advances in AI. Synthetic data, which benefits from the development and proliferation of generative models, has served as a promising substitute for real patient EHR data. However, the current generative models are limited as they only generate single type of clinical data for a synthetic patient, i.e., either continuous-valued or discrete-valued. To mimic the nature of clinical decision-making which encompasses various data types/sources, in this study, we propose a generative adversarial network (GAN) entitled EHR-M-GAN that simultaneously synthesizes mixed-type timeseries EHR data. EHR-M-GAN is capable of capturing the multidimensional, heterogeneous, and correlated temporal dynamics in patient trajectories. We have validated EHR-M-GAN on three publicly-available intensive care unit databases with records from a total of 141,488 unique patients, and performed privacy risk evaluation of the proposed model. EHR-M-GAN has demonstrated its superiority over state-of-the-art benchmarks for synthesizing clinical timeseries with high fidelity, while addressing the limitations regarding data types and dimensionality in the current generative models. Notably, prediction models for outcomes of intensive care performed significantly better when training data was augmented with the addition of EHR-M-GAN-generated timeseries. EHR-M-GAN may have use in developing AI algorithms in resource-limited settings, lowering the barrier for data acquisition while preserving patient privacy.
Collapse
Affiliation(s)
- Jin Li
- Engineering Research Center of EMR and Intelligent Expert System, Ministry of Education, College of Biomedical Engineering & Instrument Science, Zhejiang University, Hangzhou, China
- Department of Engineering Science, University of Oxford, Oxford, UK
| | - Benjamin J Cairns
- Clinical Trial Service Unit and Epidemiological Studies, Nuffield Department of Population Health, University of Oxford, Oxford, UK
| | - Jingsong Li
- Engineering Research Center of EMR and Intelligent Expert System, Ministry of Education, College of Biomedical Engineering & Instrument Science, Zhejiang University, Hangzhou, China.
- Research Center for Healthcare Data Science, Zhejiang Lab, Hangzhou, China.
| | - Tingting Zhu
- Department of Engineering Science, University of Oxford, Oxford, UK.
| |
Collapse
|
25
|
Clift AK, Dodwell D, Lord S, Petrou S, Brady M, Collins GS, Hippisley-Cox J. Development and internal-external validation of statistical and machine learning models for breast cancer prognostication: cohort study. BMJ 2023; 381:e073800. [PMID: 37164379 PMCID: PMC10170264 DOI: 10.1136/bmj-2022-073800] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 03/28/2023] [Indexed: 05/12/2023]
Abstract
OBJECTIVE To develop a clinically useful model that estimates the 10 year risk of breast cancer related mortality in women (self-reported female sex) with breast cancer of any stage, comparing results from regression and machine learning approaches. DESIGN Population based cohort study. SETTING QResearch primary care database in England, with individual level linkage to the national cancer registry, Hospital Episodes Statistics, and national mortality registers. PARTICIPANTS 141 765 women aged 20 years and older with a diagnosis of invasive breast cancer between 1 January 2000 and 31 December 2020. MAIN OUTCOME MEASURES Four model building strategies comprising two regression (Cox proportional hazards and competing risks regression) and two machine learning (XGBoost and an artificial neural network) approaches. Internal-external cross validation was used for model evaluation. Random effects meta-analysis that pooled estimates of discrimination and calibration metrics, calibration plots, and decision curve analysis were used to assess model performance, transportability, and clinical utility. RESULTS During a median 4.16 years (interquartile range 1.76-8.26) of follow-up, 21 688 breast cancer related deaths and 11 454 deaths from other causes occurred. Restricting to 10 years maximum follow-up from breast cancer diagnosis, 20 367 breast cancer related deaths occurred during a total of 688 564.81 person years. The crude breast cancer mortality rate was 295.79 per 10 000 person years (95% confidence interval 291.75 to 299.88). Predictors varied for each regression model, but both Cox and competing risks models included age at diagnosis, body mass index, smoking status, route to diagnosis, hormone receptor status, cancer stage, and grade of breast cancer. The Cox model's random effects meta-analysis pooled estimate for Harrell's C index was the highest of any model at 0.858 (95% confidence interval 0.853 to 0.864, and 95% prediction interval 0.843 to 0.873). It appeared acceptably calibrated on calibration plots. The competing risks regression model had good discrimination: pooled Harrell's C index 0.849 (0.839 to 0.859, and 0.821 to 0.876, and evidence of systematic miscalibration on summary metrics was lacking. The machine learning models had acceptable discrimination overall (Harrell's C index: XGBoost 0.821 (0.813 to 0.828, and 0.805 to 0.837); neural network 0.847 (0.835 to 0.858, and 0.816 to 0.878)), but had more complex patterns of miscalibration and more variable regional and stage specific performance. Decision curve analysis suggested that the Cox and competing risks regression models tested may have higher clinical utility than the two machine learning approaches. CONCLUSION In women with breast cancer of any stage, using the predictors available in this dataset, regression based methods had better and more consistent performance compared with machine learning approaches and may be worthy of further evaluation for potential clinical use, such as for stratified follow-up.
Collapse
Affiliation(s)
- Ash Kieran Clift
- Cancer Research UK Oxford Centre, Oxford, UK
- Nuffield Department of Primary Care Health Sciences, Radcliffe Primary Care Building, Radcliffe Observatory Quarter, University of Oxford, Oxford OX2 6GG, UK
| | - David Dodwell
- Nuffield Department of Population Health, University of Oxford, Oxford, UK
| | - Simon Lord
- Department of Oncology, University of Oxford, Oxford, UK
| | - Stavros Petrou
- Nuffield Department of Primary Care Health Sciences, Radcliffe Primary Care Building, Radcliffe Observatory Quarter, University of Oxford, Oxford OX2 6GG, UK
| | - Michael Brady
- Department of Oncology, University of Oxford, Oxford, UK
| | - Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Julia Hippisley-Cox
- Nuffield Department of Primary Care Health Sciences, Radcliffe Primary Care Building, Radcliffe Observatory Quarter, University of Oxford, Oxford OX2 6GG, UK
| |
Collapse
|
26
|
Fraser AG, Biasin E, Bijnens B, Bruining N, Caiani EG, Cobbaert K, Davies RH, Gilbert SH, Hovestadt L, Kamenjasevic E, Kwade Z, McGauran G, O'Connor G, Vasey B, Rademakers FE. Artificial intelligence in medical device software and high-risk medical devices - a review of definitions, expert recommendations and regulatory initiatives. Expert Rev Med Devices 2023; 20:467-491. [PMID: 37157833 DOI: 10.1080/17434440.2023.2184685] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
INTRODUCTION Artificial intelligence (AI) encompasses a wide range of algorithms with risks when used to support decisions about diagnosis or treatment, so professional and regulatory bodies are recommending how they should be managed. AREAS COVERED AI systems may qualify as standalone medical device software (MDSW) or be embedded within a medical device. Within the European Union (EU) AI software must undergo a conformity assessment procedure to be approved as a medical device. The draft EU Regulation on AI proposes rules that will apply across industry sectors, while for devices the Medical Device Regulation also applies. In the CORE-MD project (Coordinating Research and Evidence for Medical Devices), we have surveyed definitions and summarize initiatives made by professional consensus groups, regulators, and standardization bodies. EXPERT OPINION The level of clinical evidence required should be determined according to each application and to legal and methodological factors that contribute to risk, including accountability, transparency, and interpretability. EU guidance for MDSW based on international recommendations does not yet describe the clinical evidence needed for medical AI software. Regulators, notified bodies, manufacturers, clinicians and patients would all benefit from common standards for the clinical evaluation of high-risk AI applications and transparency of their evidence and performance.
Collapse
Affiliation(s)
- Alan G Fraser
- University Hospital of Wales, School of Medicine, Cardiff University, Heath Park, Cardiff, U.K
- KU Leuven, Leuven, Belgium
| | | | - Bart Bijnens
- Engineering Sciences, Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
| | - Nico Bruining
- Department of Clinical and Experimental Information processing (Digital Cardiology), Erasmus Medical Center, Thoraxcenter, Rotterdam, the Netherlands
| | - Enrico G Caiani
- Department of Electronics, Information and Biomedical Engineering, Politecnico di Milano, Milan, Italy
| | | | - Rhodri H Davies
- Institute of Cardiovascular Science, University College London, London, U.K
| | - Stephen H Gilbert
- Technische Universität Dresden, Else Kröner Fresenius Center for Digital Health, Dresden, Germany
| | | | | | | | | | | | - Baptiste Vasey
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK
| | | |
Collapse
|
27
|
Truchot A, Raynaud M, Kamar N, Naesens M, Legendre C, Delahousse M, Thaunat O, Buchler M, Crespo M, Linhares K, Orandi BJ, Akalin E, Pujol GS, Silva HT, Gupta G, Segev DL, Jouven X, Bentall AJ, Stegall MD, Lefaucheur C, Aubert O, Loupy A. Machine learning does not outperform traditional statistical modelling for kidney allograft failure prediction. Kidney Int 2023; 103:936-948. [PMID: 36572246 DOI: 10.1016/j.kint.2022.12.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Revised: 11/04/2022] [Accepted: 12/15/2022] [Indexed: 12/24/2022]
Abstract
Machine learning (ML) models have recently shown potential for predicting kidney allograft outcomes. However, their ability to outperform traditional approaches remains poorly investigated. Therefore, using large cohorts of kidney transplant recipients from 14 centers worldwide, we developed ML-based prediction models for kidney allograft survival and compared their prediction performances to those achieved by a validated Cox-Based Prognostication System (CBPS). In a French derivation cohort of 4000 patients, candidate determinants of allograft failure including donor, recipient and transplant-related parameters were used as predictors to develop tree-based models (RSF, RSF-ERT, CIF), Support Vector Machine models (LK-SVM, AK-SVM) and a gradient boosting model (XGBoost). Models were externally validated with cohorts of 2214 patients from Europe, 1537 from North America, and 671 from South America. Among these 8422 kidney transplant recipients, 1081 (12.84%) lost their grafts after a median post-transplant follow-up time of 6.25 years (Inter Quartile Range 4.33-8.73). At seven years post-risk evaluation, the ML models achieved a C-index of 0.788 (95% bootstrap percentile confidence interval 0.736-0.833), 0.779 (0.724-0.825), 0.786 (0.735-0.832), 0.527 (0.456-0.602), 0.704 (0.648-0.759) and 0.767 (0.711-0.815) for RSF, RSF-ERT, CIF, LK-SVM, AK-SVM and XGBoost respectively, compared with 0.808 (0.792-0.829) for the CBPS. In validation cohorts, ML models' discrimination performances were in a similar range of those of the CBPS. Calibrations of the ML models were similar or less accurate than those of the CBPS. Thus, when using a transparent methodological pipeline in validated international cohorts, ML models, despite overall good performances, do not outperform a traditional CBPS in predicting kidney allograft failure. Hence, our current study supports the continued use of traditional statistical approaches for kidney graft prognostication.
Collapse
Affiliation(s)
- Agathe Truchot
- Université de Paris, INSERM, PARCC, Paris Translational Research Centre for Organ Transplantation, Paris, France
| | - Marc Raynaud
- Université de Paris, INSERM, PARCC, Paris Translational Research Centre for Organ Transplantation, Paris, France
| | - Nassim Kamar
- Université Paul Sabatier, INSERM, Department of Nephrology and Organ Transplantation, CHU Rangueil and Purpan, Toulouse, France
| | - Maarten Naesens
- Department of Microbiology, Immunology and Transplantation, KU Leuven, Leuven, Belgium
| | - Christophe Legendre
- Université de Paris, INSERM, PARCC, Paris Translational Research Centre for Organ Transplantation, Paris, France; Kidney Transplant Department, Necker Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
| | - Michel Delahousse
- Department of Transplantation, Nephrology and Clinical Immunology, Foch Hospital, Suresnes, France
| | - Olivier Thaunat
- Department of Transplantation, Nephrology and Clinical Immunology, Hospices Civils de Lyon, Lyon, France
| | - Matthias Buchler
- Nephrology and Immunology Department, Bretonneau Hospital, Tours, France
| | - Marta Crespo
- Department of Nephrology, Hospital del Mar Barcelona, Barcelona, Spain
| | - Kamilla Linhares
- Hospital do Rim, Escola Paulista de Medicina, Universidade Federal de São Paulo, São Paulo, Brazil
| | - Babak J Orandi
- University of Alabama at Birmingham Heersink School of Medicine, Birmingham, Alabama, USA
| | - Enver Akalin
- Renal Division, Montefiore Medical Centre, Kidney Transplantation Program, Albert Einstein College of Medicine, New York, New York, USA
| | - Gervacio Soler Pujol
- Unidad de Trasplante Renopancreas, Centro de Educacion Medica e Investigaciones Clinicas Buenos Aires, Buenos Aires, Argentina
| | - Helio Tedesco Silva
- Hospital do Rim, Escola Paulista de Medicina, Universidade Federal de São Paulo, São Paulo, Brazil
| | - Gaurav Gupta
- Division of Nephrology, Department of Internal Medicine, Virginia Commonwealth University School of Medicine, Richmond, Virginia, USA
| | - Dorry L Segev
- Department of Surgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
| | - Xavier Jouven
- Université de Paris, INSERM, PARCC, Paris Translational Research Centre for Organ Transplantation, Paris, France; Cardiology Department, European Georges Pompidou Hospital, Paris, France
| | - Andrew J Bentall
- William J von Liebig Centre for Transplantation and Clinical Regeneration, Mayo Clinic, Rochester, Minnesota, USA
| | - Mark D Stegall
- William J von Liebig Centre for Transplantation and Clinical Regeneration, Mayo Clinic, Rochester, Minnesota, USA
| | - Carmen Lefaucheur
- Université de Paris, INSERM, PARCC, Paris Translational Research Centre for Organ Transplantation, Paris, France; Kidney Transplant Department, Saint-Louis Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
| | - Olivier Aubert
- Université de Paris, INSERM, PARCC, Paris Translational Research Centre for Organ Transplantation, Paris, France; Kidney Transplant Department, Necker Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
| | - Alexandre Loupy
- Université de Paris, INSERM, PARCC, Paris Translational Research Centre for Organ Transplantation, Paris, France; Kidney Transplant Department, Necker Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France.
| |
Collapse
|
28
|
Pham N, Hill V, Rauschecker A, Lui Y, Niogi S, Fillipi CG, Chang P, Zaharchuk G, Wintermark M. Critical Appraisal of Artificial Intelligence-Enabled Imaging Tools Using the Levels of Evidence System. AJNR Am J Neuroradiol 2023; 44:E21-E28. [PMID: 37080722 PMCID: PMC10171388 DOI: 10.3174/ajnr.a7850] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Accepted: 03/16/2023] [Indexed: 04/22/2023]
Abstract
Clinical adoption of an artificial intelligence-enabled imaging tool requires critical appraisal of its life cycle from development to implementation by using a systematic, standardized, and objective approach that can verify both its technical and clinical efficacy. Toward this concerted effort, the ASFNR/ASNR Artificial Intelligence Workshop Technology Working Group is proposing a hierarchal evaluation system based on the quality, type, and amount of scientific evidence that the artificial intelligence-enabled tool can demonstrate for each component of its life cycle. The current proposal is modeled after the levels of evidence in medicine, with the uppermost level of the hierarchy showing the strongest evidence for potential impact on patient care and health care outcomes. The intended goal of establishing an evidence-based evaluation system is to encourage transparency, foster an understanding of the creation of artificial intelligence tools and the artificial intelligence decision-making process, and to report the relevant data on the efficacy of artificial intelligence tools that are developed. The proposed system is an essential step in working toward a more formalized, clinically validated, and regulated framework for the safe and effective deployment of artificial intelligence imaging applications that will be used in clinical practice.
Collapse
Affiliation(s)
- N Pham
- From the Department of Radiology (N.P., G.Z.), Stanford School of Medicine, Palo Alto, California
| | - V Hill
- Department of Radiology (V.H.), Northwestern University Feinberg School of Medicine, Chicago, Illinois
| | - A Rauschecker
- Department of Radiology (A.R.), University of California, San Francisco, San Francisco, California
| | - Y Lui
- Department of Radiology (Y.L.), NYU Grossman School of Medicine, New York, New York
| | - S Niogi
- Department of Radiology (S.N.), Weill Cornell Medicine, New York, New York
| | - C G Fillipi
- Department of Radiology (C.G.F.), Tufts University School of Medicine, Boston, Massachusetts
| | - P Chang
- Department of Radiology (P.C.), University of California, Irvine, Irvine, California
| | - G Zaharchuk
- From the Department of Radiology (N.P., G.Z.), Stanford School of Medicine, Palo Alto, California
| | - M Wintermark
- Department of Neuroradiology (M.W.), The University of Texas MD Anderson Cancer Center, Houston, Texas
| |
Collapse
|
29
|
Boussina A, Shashikumar S, Amrollahi F, Pour H, Hogarth M, Nemati S. Development & Deployment of a Real-time Healthcare Predictive Analytics Platform. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.04.10.23288373. [PMID: 37090509 PMCID: PMC10120788 DOI: 10.1101/2023.04.10.23288373] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/25/2023]
Abstract
The deployment of predictive analytic algorithms that can safely and seamlessly integrate into existing healthcare workflows remains a significant challenge. Here, we present a scalable, cloud-based, fault-tolerant platform that is capable of extracting and processing electronic health record (EHR) data for any patient at any time following admission and transferring results back into the EHR. This platform has been successfully deployed within the UC San Diego Health system and utilizes interoperable data standards to enable portability.
Collapse
Affiliation(s)
- Aaron Boussina
- Division of Biomedical Informatics, University of California San Diego, San Diego, USA
| | - Supreeth Shashikumar
- Division of Biomedical Informatics, University of California San Diego, San Diego, USA
| | - Fatemeh Amrollahi
- Division of Biomedical Informatics, University of California San Diego, San Diego, USA
| | - Hayden Pour
- Division of Biomedical Informatics, University of California San Diego, San Diego, USA
| | - Michael Hogarth
- Division of Biomedical Informatics, University of California San Diego, San Diego, USA
| | - Shamim Nemati
- Division of Biomedical Informatics, University of California San Diego, San Diego, USA
| |
Collapse
|
30
|
Oh M, Zhang L. DeepGeni: deep generalized interpretable autoencoder elucidates gut microbiota for better cancer immunotherapy. Sci Rep 2023; 13:4599. [PMID: 36944780 PMCID: PMC10030841 DOI: 10.1038/s41598-023-31210-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2022] [Accepted: 03/08/2023] [Indexed: 03/23/2023] Open
Abstract
Recent studies revealed that gut microbiota modulates the response to cancer immunotherapy and fecal microbiota transplantation has clinical benefits in melanoma patients during treatment. Understanding how microbiota affects individual responses is crucial for precision oncology. However, it is challenging to identify key microbial taxa with limited data as statistical and machine learning models often lose their generalizability. In this study, DeepGeni, a deep generalized interpretable autoencoder, is proposed to improve the generalizability and interpretability of microbiome profiles by augmenting data and by introducing interpretable links in the autoencoder. DeepGeni-based machine learning classifier outperforms state-of-the-art classifier in the microbiome-driven prediction of responsiveness of melanoma patients treated with immune checkpoint inhibitors. Moreover, the interpretable links of DeepGeni elucidate the most informative microbiota associated with cancer immunotherapy response. DeepGeni not only improves microbiome-driven prediction of immune checkpoint inhibitor responsiveness but also suggests potential microbial targets for fecal microbiota transplant or probiotics improving the outcome of cancer immunotherapy.
Collapse
Affiliation(s)
- Min Oh
- Department of Computer Science, Virginia Tech, Blacksburg, VA, USA
- Microsoft Research, Redmond, WA, USA
| | - Liqing Zhang
- Department of Computer Science, Virginia Tech, Blacksburg, VA, USA.
| |
Collapse
|
31
|
Machine Learning Models to Forecast Outcomes of Pituitary Surgery: A Systematic Review in Quality of Reporting and Current Evidence. Brain Sci 2023; 13:brainsci13030495. [PMID: 36979305 PMCID: PMC10046799 DOI: 10.3390/brainsci13030495] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Revised: 03/08/2023] [Accepted: 03/13/2023] [Indexed: 03/17/2023] Open
Abstract
Background: The complex nature and heterogeneity involving pituitary surgery results have increased interest in machine learning (ML) applications for prediction of outcomes over the last decade. This study aims to systematically review the characteristics of ML models involving pituitary surgery outcome prediction and assess their reporting quality. Methods: We searched the PubMed, Scopus, and Web of Knowledge databases for publications on the use of ML to predict pituitary surgery outcomes. We used the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) to assess report quality. Our search strategy was based on the terms “artificial intelligence”, “machine learning”, and “pituitary”. Results: 20 studies were included in this review. The principal models reported in each article were post-surgical endocrine outcomes (n = 10), tumor management (n = 3), and intra- and postoperative complications (n = 7). Overall, the included studies adhered to a median of 65% (IQR = 60–72%) of TRIPOD criteria, ranging from 43% to 83%. The median reported AUC was 0.84 (IQR = 0.80–0.91). The most popular algorithms were support vector machine (n = 5) and random forest (n = 5). Only two studies reported external validation and adherence to any reporting guideline. Calibration methods were not reported in 15 studies. No model achieved the phase of actual clinical applicability. Conclusion: Applications of ML in the prediction of pituitary outcomes are still nascent, as evidenced by the lack of any model validated for clinical practice. Although studies have demonstrated promising results, greater transparency in model development and reporting is needed to enable their use in clinical practice. Further adherence to reporting guidelines can help increase AI’s real-world utility and improve clinical practice.
Collapse
|
32
|
Hampel H, Gao P, Cummings J, Toschi N, Thompson PM, Hu Y, Cho M, Vergallo A. The foundation and architecture of precision medicine in neurology and psychiatry. Trends Neurosci 2023; 46:176-198. [PMID: 36642626 PMCID: PMC10720395 DOI: 10.1016/j.tins.2022.12.004] [Citation(s) in RCA: 32] [Impact Index Per Article: 32.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Revised: 11/18/2022] [Accepted: 12/14/2022] [Indexed: 01/15/2023]
Abstract
Neurological and psychiatric diseases have high degrees of genetic and pathophysiological heterogeneity, irrespective of clinical manifestations. Traditional medical paradigms have focused on late-stage syndromic aspects of these diseases, with little consideration of the underlying biology. Advances in disease modeling and methodological design have paved the way for the development of precision medicine (PM), an established concept in oncology with growing attention from other medical specialties. We propose a PM architecture for central nervous system diseases built on four converging pillars: multimodal biomarkers, systems medicine, digital health technologies, and data science. We discuss Alzheimer's disease (AD), an area of significant unmet medical need, as a case-in-point for the proposed framework. AD can be seen as one of the most advanced PM-oriented disease models and as a compelling catalyzer towards PM-oriented neuroscience drug development and advanced healthcare practice.
Collapse
Affiliation(s)
- Harald Hampel
- Alzheimer's Disease & Brain Health, Eisai Inc., Nutley, NJ, USA.
| | - Peng Gao
- Alzheimer's Disease & Brain Health, Eisai Inc., Nutley, NJ, USA
| | - Jeffrey Cummings
- Chambers-Grundy Center for Transformative Neuroscience, Department of Brain Health, School of Integrated Health Sciences, University of Nevada Las Vegas (UNLV), Las Vegas, NV, USA
| | - Nicola Toschi
- Department of Biomedicine and Prevention, University of Rome Tor Vergata, Rome, Italy; Athinoula A. Martinos Center for Biomedical Imaging and Harvard Medical School, Boston, MA, USA
| | - Paul M Thompson
- Imaging Genetics Center, Mark & Mary Stevens Institute for Neuroimaging & Informatics, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA
| | - Yan Hu
- Alzheimer's Disease & Brain Health, Eisai Inc., Nutley, NJ, USA
| | - Min Cho
- Alzheimer's Disease & Brain Health, Eisai Inc., Nutley, NJ, USA
| | - Andrea Vergallo
- Alzheimer's Disease & Brain Health, Eisai Inc., Nutley, NJ, USA
| |
Collapse
|
33
|
Gumz ML, Shimbo D, Abdalla M, Balijepalli RC, Benedict C, Chen Y, Earnest DJ, Gamble KL, Garrison SR, Gong MC, Hogenesch JB, Hong Y, Ivy JR, Joe B, Laposky AD, Liang M, MacLaughlin EJ, Martino TA, Pollock DM, Redline S, Rogers A, Dan Rudic R, Schernhammer ES, Stergiou GS, St-Onge MP, Wang X, Wright J, Oh YS. Toward Precision Medicine: Circadian Rhythm of Blood Pressure and Chronotherapy for Hypertension - 2021 NHLBI Workshop Report. Hypertension 2023; 80:503-522. [PMID: 36448463 PMCID: PMC9931676 DOI: 10.1161/hypertensionaha.122.19372] [Citation(s) in RCA: 24] [Impact Index Per Article: 24.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/02/2022]
Abstract
Healthy individuals exhibit blood pressure variation over a 24-hour period with higher blood pressure during wakefulness and lower blood pressure during sleep. Loss or disruption of the blood pressure circadian rhythm has been linked to adverse health outcomes, for example, cardiovascular disease, dementia, and chronic kidney disease. However, the current diagnostic and therapeutic approaches lack sufficient attention to the circadian rhythmicity of blood pressure. Sleep patterns, hormone release, eating habits, digestion, body temperature, renal and cardiovascular function, and other important host functions as well as gut microbiota exhibit circadian rhythms, and influence circadian rhythms of blood pressure. Potential benefits of nonpharmacologic interventions such as meal timing, and pharmacologic chronotherapeutic interventions, such as the bedtime administration of antihypertensive medications, have recently been suggested in some studies. However, the mechanisms underlying circadian rhythm-mediated blood pressure regulation and the efficacy of chronotherapy in hypertension remain unclear. This review summarizes the results of the National Heart, Lung, and Blood Institute workshop convened on October 27 to 29, 2021 to assess knowledge gaps and research opportunities in the study of circadian rhythm of blood pressure and chronotherapy for hypertension.
Collapse
Affiliation(s)
- Michelle L Gumz
- Department of Physiology and Aging; Center for Integrative Cardiovascular and Metabolic Disease, Department of Medicine, Division of Nephrology, Hypertension and Renal Transplantation, University of Florida, Gainesville, FL (M.L.G.)
| | - Daichi Shimbo
- Department of Medicine, The Columbia Hypertension Center, Columbia University Irving Medical Center, New York, NY (D.S.)
| | - Marwah Abdalla
- Department of Medicine, Center for Behavioral Cardiovascular Health, Columbia University Irving Medical Center, New York, NY (M.A.)
| | - Ravi C Balijepalli
- Division of Cardiovascular Sciences, National Heart, Lung, and Blood Institute, NIH, Bethesda, MD (R.C.B., Y.H., J.W., Y.S.O.)
| | - Christian Benedict
- Department of Pharmaceutical Biosciences, Molecular Neuropharmacology, Uppsala University, Sweden (C.B.)
| | - Yabing Chen
- Department of Pathology, University of Alabama at Birmingham, and Research Department, Birmingham VA Medical Center, AL (Y.C.)
| | - David J Earnest
- Department of Neuroscience & Experimental Therapeutics, Texas A&M University, Bryan, TX (D.J.E.)
| | - Karen L Gamble
- Department of Psychiatry and Behavioral Neurobiology, University of Alabama at Birmingham, AL (K.L.G.)
| | - Scott R Garrison
- Department of Family Medicine, University of Alberta, Canada (S.R.G.)
| | - Ming C Gong
- Department of Physiology, University of Kentucky, Lexington, KY (M.C.G.)
| | | | - Yuling Hong
- Division of Cardiovascular Sciences, National Heart, Lung, and Blood Institute, NIH, Bethesda, MD (R.C.B., Y.H., J.W., Y.S.O.)
| | - Jessica R Ivy
- University/British Heart Foundation Centre for Cardiovascular Science, The Queen's Medical Research Institute, The University of Edinburgh, United Kingdom (J.R.I.)
| | - Bina Joe
- Department of Physiology and Pharmacology and Center for Hypertension and Precision Medicine, University of Toledo College of Medicine and Life Sciences, OH (B.J.)
| | - Aaron D Laposky
- National Center on Sleep Disorders Research, Division of Lung Diseases, National Heart, Lung, and Blood Institute, NIH, Bethesda, MD (A.D.L.)
| | - Mingyu Liang
- Center of Systems Molecular Medicine, Department of Physiology, Medical College of Wisconsin, Milwaukee, WI (M.L.)
| | - Eric J MacLaughlin
- Department of Pharmacy Practice, Texas Tech University Health Sciences Center, Amarillo, TX (E.J.M.)
| | - Tami A Martino
- Center for Cardiovascular Investigations, Department of Biomedical Sciences, University of Guelph, Ontario, Canada (T.A.M.)
| | - David M Pollock
- Division of Nephrology, Department of Medicine, University of Alabama at Birmingham, AL (D.M.P.)
| | - Susan Redline
- Division of Sleep and Circadian Disorders, Brigham and Women's Hospital, Harvard Medical School, Boston, MA (S.R.)
| | - Amy Rogers
- Division of Molecular and Clinical Medicine, University of Dundee, United Kingdom (A.R.)
| | - R Dan Rudic
- Department of Pharmacology and Toxicology, Augusta University, GA (R.D.R.)
| | - Eva S Schernhammer
- Channing Division of Network Medicine, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, MA (E.S.S.)
| | - George S Stergiou
- Hypertension Center, STRIDE-7, National and Kapodistrian University of Athens, School of Medicine, Third Department of Medicine, Sotiria Hospital, Athens, Greece (G.S.S.)
| | - Marie-Pierre St-Onge
- Division of General Medicine, Department of Medicine, Columbia University Irving Medical Center' New York, NY (M.-P.S.-O.)
| | - Xiaoling Wang
- Georgia Prevention Institute, Department of Medicine, Augusta University, GA (X.W.)
| | - Jacqueline Wright
- Division of Cardiovascular Sciences, National Heart, Lung, and Blood Institute, NIH, Bethesda, MD (R.C.B., Y.H., J.W., Y.S.O.)
| | - Young S Oh
- Division of Cardiovascular Sciences, National Heart, Lung, and Blood Institute, NIH, Bethesda, MD (R.C.B., Y.H., J.W., Y.S.O.)
| |
Collapse
|
34
|
Sufriyana H, Wu YW, Su ECY. Human-guided deep learning with ante-hoc explainability by convolutional network from non-image data for pregnancy prognostication. Neural Netw 2023; 162:99-116. [PMID: 36898257 DOI: 10.1016/j.neunet.2023.02.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 01/30/2023] [Accepted: 02/14/2023] [Indexed: 02/26/2023]
Abstract
BACKGROUND AND OBJECTIVE Deep learning is applied in medicine mostly due to its state-of-the-art performance for diagnostic imaging. Supervisory authorities also require the model to be explainable, but most explain the model after development (post hoc) instead of incorporating explanation into the design (ante hoc). This study aimed to demonstrate a human-guided deep learning with ante-hoc explainability by convolutional network from non-image data to develop, validate, and deploy a prognostic prediction model for PROM and an estimator of time of delivery using a nationwide health insurance database. METHODS To guide modeling, we constructed and verified association diagrams respectively from literatures and electronic health records. Non-image data were transformed into meaningful images utilizing predictor-to-predictor similarities, harnessing the power of convolutional neural network mostly used for diagnostic imaging. The network architecture was also inferred from the similarities. RESULTS This resulted the best model for prelabor rupture of membranes (n=883, 376) with the area under curves 0.73 (95% CI 0.72 to 0.75) and 0.70 (95% CI 0.69 to 0.71) respectively by internal and external validations, and outperformed previous models found by systematic review. It was explainable by knowledge-based diagrams and model representation. CONCLUSIONS This allows prognostication with actionable insights for preventive medicine.
Collapse
Affiliation(s)
- Herdiantri Sufriyana
- Graduate Institute of Biomedical Informatics, College of Medical Science and Technology, Taipei Medical University, 250 Wu-Xing Street, Taipei 11031, Taiwan; Department of Medical Physiology, Faculty of Medicine, Universitas Nahdlatul Ulama Surabaya, 57 Raya Jemursari Road, Surabaya 60237, Indonesia
| | - Yu-Wei Wu
- Graduate Institute of Biomedical Informatics, College of Medical Science and Technology, Taipei Medical University, 250 Wu-Xing Street, Taipei 11031, Taiwan; Clinical Big Data Research Center, Taipei Medical University Hospital, 250 Wu-Xing Street, Taipei 11031, Taiwan
| | - Emily Chia-Yu Su
- Graduate Institute of Biomedical Informatics, College of Medical Science and Technology, Taipei Medical University, 250 Wu-Xing Street, Taipei 11031, Taiwan; Clinical Big Data Research Center, Taipei Medical University Hospital, 250 Wu-Xing Street, Taipei 11031, Taiwan; Research Center for Artificial Intelligence in Medicine, Taipei Medical University, 250 Wu-Xing Street, Taipei 11031, Taiwan.
| |
Collapse
|
35
|
Wu T, Wei Y, Wu J, Yi B, Li H. Logistic regression technique is comparable to complex machine learning algorithms in predicting cognitive impairment related to post intensive care syndrome. Sci Rep 2023; 13:2485. [PMID: 36774378 PMCID: PMC9922285 DOI: 10.1038/s41598-023-28421-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Accepted: 01/18/2023] [Indexed: 02/13/2023] Open
Abstract
To evaluate the performance of machine learning (ML) models and to compare it with logistic regression (LR) technique in predicting cognitive impairment related to post intensive care syndrome (PICS-CI). We conducted a prospective observational study of ICU patients at two tertiary hospitals. A cohort of 2079 patients was screened, and finally 481 patients were included. Seven different ML models were considered, decision tree (DT), random forest (RF), XGBoost, neural network (NN), naïve bayes (NB), and support vector machine (SVM), and compared with logistic regression (LR). Discriminative ability was evaluated by area under the receiver operating characteristic curve (AUC), calibration belt plots, and Hosmer-Lemeshow test was used to assess calibration. Decision curve analysis was performed to quantify clinical utility. Duration of delirium, poor Richards-Campbell sleep questionnaire (RCSQ) score, advanced age, and sepsis were the most frequent and important candidates risk factors for PICS-CI. All ML models showed good performance (AUC range: 0.822-0.906). NN model had the highest AUC (0.906 [95% CI 0.857-0.955]), which was slightly higher than, but not significantly different from that of LR (0.898 [95% CI 0.847-0.949]) (P > 0.05, Delong test). Given the overfitting and complexity of some ML models, the LR model was then used to develop a web-based risk calculator to aid decision-making ( https://model871010.shinyapps.io/dynnomapp/ ). In a low dimensional data, LR may yield as good performance as other complex ML models to predict cognitive impairment after ICU hospitalization.
Collapse
Affiliation(s)
- TingTing Wu
- The School of Nursing, Fujian Medical University, Fujian, China
- Research Center for Nursing Theory and Practice, Fujian Provincial Hospital, No 134, East Street, Gulou District, Fuzhou, 35001, Fujian, China
| | - YueQing Wei
- Research Center for Nursing Theory and Practice, Fujian Provincial Hospital, No 134, East Street, Gulou District, Fuzhou, 35001, Fujian, China
- Respiratory and Intensive Care Unit, Fujian Provincial Hospital, Fujian, China
| | - JingBing Wu
- Research Center for Nursing Theory and Practice, Fujian Provincial Hospital, No 134, East Street, Gulou District, Fuzhou, 35001, Fujian, China
- Medical Intensive Care Unit, Fujian Provincial Hospital, Fujian, China
| | - BiLan Yi
- Research Center for Nursing Theory and Practice, Fujian Provincial Hospital, No 134, East Street, Gulou District, Fuzhou, 35001, Fujian, China
- Respiratory and Intensive Care Unit, Fujian Provincial Hospital, Fujian, China
| | - Hong Li
- The School of Nursing, Fujian Medical University, Fujian, China.
- Research Center for Nursing Theory and Practice, Fujian Provincial Hospital, No 134, East Street, Gulou District, Fuzhou, 35001, Fujian, China.
| |
Collapse
|
36
|
Chen ZS, Kulkarni P(P, Galatzer-Levy IR, Bigio B, Nasca C, Zhang Y. Modern views of machine learning for precision psychiatry. PATTERNS (NEW YORK, N.Y.) 2022; 3:100602. [PMID: 36419447 PMCID: PMC9676543 DOI: 10.1016/j.patter.2022.100602] [Citation(s) in RCA: 21] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
Abstract
In light of the National Institute of Mental Health (NIMH)'s Research Domain Criteria (RDoC), the advent of functional neuroimaging, novel technologies and methods provide new opportunities to develop precise and personalized prognosis and diagnosis of mental disorders. Machine learning (ML) and artificial intelligence (AI) technologies are playing an increasingly critical role in the new era of precision psychiatry. Combining ML/AI with neuromodulation technologies can potentially provide explainable solutions in clinical practice and effective therapeutic treatment. Advanced wearable and mobile technologies also call for the new role of ML/AI for digital phenotyping in mobile mental health. In this review, we provide a comprehensive review of ML methodologies and applications by combining neuroimaging, neuromodulation, and advanced mobile technologies in psychiatry practice. We further review the role of ML in molecular phenotyping and cross-species biomarker identification in precision psychiatry. We also discuss explainable AI (XAI) and neuromodulation in a closed human-in-the-loop manner and highlight the ML potential in multi-media information extraction and multi-modal data fusion. Finally, we discuss conceptual and practical challenges in precision psychiatry and highlight ML opportunities in future research.
Collapse
Affiliation(s)
- Zhe Sage Chen
- Department of Psychiatry, New York University Grossman School of Medicine, New York, NY 10016, USA
- Department of Neuroscience and Physiology, New York University Grossman School of Medicine, New York, NY 10016, USA
- The Neuroscience Institute, New York University Grossman School of Medicine, New York, NY 10016, USA
- Department of Biomedical Engineering, New York University Tandon School of Engineering, Brooklyn, NY 11201, USA
| | | | - Isaac R. Galatzer-Levy
- Department of Psychiatry, New York University Grossman School of Medicine, New York, NY 10016, USA
- Meta Reality Lab, New York, NY, USA
| | - Benedetta Bigio
- Department of Psychiatry, New York University Grossman School of Medicine, New York, NY 10016, USA
| | - Carla Nasca
- Department of Psychiatry, New York University Grossman School of Medicine, New York, NY 10016, USA
- The Neuroscience Institute, New York University Grossman School of Medicine, New York, NY 10016, USA
| | - Yu Zhang
- Department of Bioengineering, Lehigh University, Bethlehem, PA 18015, USA
- Department of Electrical and Computer Engineering, Lehigh University, Bethlehem, PA 18015, USA
| |
Collapse
|
37
|
Koutsouleris N, Hauser TU, Skvortsova V, De Choudhury M. From promise to practice: towards the realisation of AI-informed mental health care. THE LANCET DIGITAL HEALTH 2022; 4:e829-e840. [DOI: 10.1016/s2589-7500(22)00153-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/28/2022] [Revised: 07/14/2022] [Accepted: 07/27/2022] [Indexed: 11/07/2022]
|
38
|
Olusanya MO, Ogunsakin RE, Ghai M, Adeleke MA. Accuracy of Machine Learning Classification Models for the Prediction of Type 2 Diabetes Mellitus: A Systematic Survey and Meta-Analysis Approach. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:ijerph192114280. [PMID: 36361161 PMCID: PMC9655196 DOI: 10.3390/ijerph192114280] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/27/2022] [Revised: 10/22/2022] [Accepted: 10/25/2022] [Indexed: 05/13/2023]
Abstract
Soft-computing and statistical learning models have gained substantial momentum in predicting type 2 diabetes mellitus (T2DM) disease. This paper reviews recent soft-computing and statistical learning models in T2DM using a meta-analysis approach. We searched for papers using soft-computing and statistical learning models focused on T2DM published between 2010 and 2021 on three different search engines. Of 1215 studies identified, 34 with 136952 patients met our inclusion criteria. The pooled algorithm's performance was able to predict T2DM with an overall accuracy of 0.86 (95% confidence interval [CI] of [0.82, 0.89]). The classification of diabetes prediction was significantly greater in models with a screening and diagnosis (pooled proportion [95% CI] = 0.91 [0.74, 0.97]) when compared to models with nephropathy (pooled proportion = 0.48 [0.76, 0.89] to 0.88 [0.83, 0.91]). For the prediction of T2DM, the decision trees (DT) models had a pooled accuracy of 0.88 [95% CI: 0.82, 0.92], and the neural network (NN) models had a pooled accuracy of 0.85 [95% CI: 0.79, 0.89]. Meta-regression did not provide any statistically significant findings for the heterogeneous accuracy in studies with different diabetes predictions, sample sizes, and impact factors. Additionally, ML models showed high accuracy for the prediction of T2DM. The predictive accuracy of ML algorithms in T2DM is promising, mainly through DT and NN models. However, there is heterogeneity among ML models. We compared the results and models and concluded that this evidence might help clinicians interpret data and implement optimum models for their dataset for T2DM prediction.
Collapse
Affiliation(s)
- Micheal O. Olusanya
- Department of Computer Science and Information Technology, Sol Plaatje University, Kimberley 8300, South Africa
- Correspondence:
| | - Ropo Ebenezer Ogunsakin
- Biostatistics Unit, Discipline of Public Health Medicine, School of Nursing & Public Health, College of Health Sciences, University of KwaZulu-Natal, Durban 4000, South Africa
| | - Meenu Ghai
- Discipline of Genetics, School of Life Sciences, University of KwaZulu-Natal, Durban 4000, South Africa
| | - Matthew Adekunle Adeleke
- Discipline of Genetics, School of Life Sciences, University of KwaZulu-Natal, Durban 4000, South Africa
| |
Collapse
|
39
|
Sun Y, Lü L, Cai YK, Lee P. Prediction of black carbon in marine engines and correlation analysis of model characteristics based on multiple machine learning algorithms. ENVIRONMENTAL SCIENCE AND POLLUTION RESEARCH INTERNATIONAL 2022; 29:78509-78525. [PMID: 35697984 DOI: 10.1007/s11356-022-20496-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/19/2021] [Accepted: 04/24/2022] [Indexed: 06/15/2023]
Abstract
Ship black carbon emissions have caused great harm to ecological environment. In order to estimate the black carbon emissions, thereby reducing the cost of black carbon experiments, here, we introduced four machine learning algorithms which are lasso regression, support vector machine, extreme gradient boosting, and artificial neural network to predict ship black carbon emissions. The prediction models were established with using the datasets acquired from similar marine engines under various steady-state conditions. The results show that SVM, XGB, and ANN have higher prediction accuracy than lasso regression, and the adjusted R2 of each model is 0.9810, 0.9850, 0.9885, and 0.6088. Although ANN shows the best prediction performance, it is inferior to SVM and XGB in terms of model stability and training cost. Then, in order to simplify the optimization process of hyperparameters and improve the prediction accuracy of the model at the same time, we use three different swarm intelligence algorithms to automatically optimize the hyperparameters of SVM and XGB. In addition, we applied mutual information to measure the correlation between the characteristics of the prediction models and black carbon concentration and found that the characteristics which related to in-cylinder combustion have a strong correlation with the black carbon concentration. The findings in this paper prove the feasibility of machine learning in ship black carbon emission prediction and could provide references for reducing ship black carbon emissions and the formulation of emission regulations.
Collapse
Affiliation(s)
- Ying Sun
- School of Naval Architecture, Ocean and Energy Power Engineering, Wuhan University of Technology, Wuhan, Hubei, 430070, People's Republic of China
| | - Lin Lü
- School of Naval Architecture, Ocean and Energy Power Engineering, Wuhan University of Technology, Wuhan, Hubei, 430070, People's Republic of China
| | - Yun-Kai Cai
- School of Naval Architecture, Ocean and Energy Power Engineering, Wuhan University of Technology, Wuhan, Hubei, 430070, People's Republic of China
| | - Peng Lee
- School of Naval Architecture, Ocean and Energy Power Engineering, Wuhan University of Technology, Wuhan, Hubei, 430070, People's Republic of China.
| |
Collapse
|
40
|
Pfob A, Lu SC, Sidey-Gibbons C. Machine learning in medicine: a practical introduction to techniques for data pre-processing, hyperparameter tuning, and model comparison. BMC Med Res Methodol 2022; 22:282. [PMID: 36319956 PMCID: PMC9624048 DOI: 10.1186/s12874-022-01758-8] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2022] [Accepted: 10/18/2022] [Indexed: 11/17/2022] Open
Abstract
BACKGROUND There is growing enthusiasm for the application of machine learning (ML) and artificial intelligence (AI) techniques to clinical research and practice. However, instructions on how to develop robust high-quality ML and AI in medicine are scarce. In this paper, we provide a practical example of techniques that facilitate the development of high-quality ML systems including data pre-processing, hyperparameter tuning, and model comparison using open-source software and data. METHODS We used open-source software and a publicly available dataset to train and validate multiple ML models to classify breast masses into benign or malignant using mammography image features and patient age. We compared algorithm predictions to the ground truth of histopathologic evaluation. We provide step-by-step instructions with accompanying code lines. FINDINGS Performance of the five algorithms at classifying breast masses as benign or malignant based on mammography image features and patient age was statistically equivalent (P > 0.05). Area under the receiver operating characteristics curve (AUROC) for the logistic regression with elastic net penalty was 0.89 (95% CI 0.85 - 0.94), for the Extreme Gradient Boosting Tree 0.88 (95% CI 0.83 - 0.93), for the Multivariate Adaptive Regression Spline algorithm 0.88 (95% CI 0.83 - 0.93), for the Support Vector Machine 0.89 (95% CI 0.84 - 0.93), and for the neural network 0.89 (95% CI 0.84 - 0.93). INTERPRETATION Our paper allows clinicians and medical researchers who are interested in using ML algorithms to understand and recreate the elements of a comprehensive ML analysis. Following our instructions may help to improve model generalizability and reproducibility in medical ML studies.
Collapse
Affiliation(s)
- André Pfob
- grid.5253.10000 0001 0328 4908Department of Obstetrics and Gynecology, University Breast Unit, Heidelberg University Hospital, Heidelberg, Germany ,grid.240145.60000 0001 2291 4776MD Anderson Center for INSPiRED Cancer Care, The University of Texas MD Anderson Cancer Center, Houston, USA
| | - Sheng-Chieh Lu
- grid.240145.60000 0001 2291 4776MD Anderson Center for INSPiRED Cancer Care, The University of Texas MD Anderson Cancer Center, Houston, USA ,grid.240145.60000 0001 2291 4776Section of Patient-Centered Analytics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030 USA
| | - Chris Sidey-Gibbons
- grid.240145.60000 0001 2291 4776MD Anderson Center for INSPiRED Cancer Care, The University of Texas MD Anderson Cancer Center, Houston, USA ,grid.240145.60000 0001 2291 4776Section of Patient-Centered Analytics, The University of Texas MD Anderson Cancer Center, Houston, TX 77030 USA
| |
Collapse
|
41
|
Fusar-Poli P, Manchia M, Koutsouleris N, Leslie D, Woopen C, Calkins ME, Dunn M, Tourneau CL, Mannikko M, Mollema T, Oliver D, Rietschel M, Reininghaus EZ, Squassina A, Valmaggia L, Kessing LV, Vieta E, Correll CU, Arango C, Andreassen OA. Ethical considerations for precision psychiatry: A roadmap for research and clinical practice. Eur Neuropsychopharmacol 2022; 63:17-34. [PMID: 36041245 DOI: 10.1016/j.euroneuro.2022.08.001] [Citation(s) in RCA: 36] [Impact Index Per Article: 18.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/13/2022] [Revised: 07/04/2022] [Accepted: 08/05/2022] [Indexed: 12/14/2022]
Abstract
Precision psychiatry is an emerging field with transformative opportunities for mental health. However, the use of clinical prediction models carries unprecedented ethical challenges, which must be addressed before accessing the potential benefits of precision psychiatry. This critical review covers multidisciplinary areas, including psychiatry, ethics, statistics and machine-learning, healthcare and academia, as well as input from people with lived experience of mental disorders, their family, and carers. We aimed to identify core ethical considerations for precision psychiatry and mitigate concerns by designing a roadmap for research and clinical practice. We identified priorities: learning from somatic medicine; identifying precision psychiatry use cases; enhancing transparency and generalizability; fostering implementation; promoting mental health literacy; communicating risk estimates; data protection and privacy; and fostering the equitable distribution of mental health care. We hope this blueprint will advance research and practice and enable people with mental health problems to benefit from precision psychiatry.
Collapse
Affiliation(s)
- Paolo Fusar-Poli
- Early Psychosis: Interventions and Clinical-Detection (EPIC) Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK; South London and Maudsley NHS Foundation Trust, London, UK; Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy.
| | - Mirko Manchia
- Section of Psychiatry, Department of Medical Sciences and Public Health, University of Cagliari, Cagliari, Italy; Unit of Clinical Psychiatry, University Hospital Agency of Cagliari, Cagliari, Italy; Department of Pharmacology, Dalhousie University, Halifax, Nova Scotia, Canada
| | | | | | | | - Monica E Calkins
- Neurodevelopment and Psychosis Section and Lifespan Brain Institute of Penn/CHOP, Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, USA
| | - Michael Dunn
- Centre for Biomedical Ethics, Yong Loo Lin School of Medicine, National University of Singapore
| | - Christophe Le Tourneau
- Institut Curie, Department of Drug Development and Innovation (D3i), INSERM U900 Research unit, Paris-Saclay University, France
| | - Miia Mannikko
- European Federation of Associations of Families of People with Mental Illness (EUFAMI), Leuven, Belgium
| | - Tineke Mollema
- Global Alliance of Mental Illness Advocacy Networks-Europe (GAMIAN), Brussels, Belgium
| | - Dominic Oliver
- Early Psychosis: Interventions and Clinical-Detection (EPIC) Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK
| | - Marcella Rietschel
- Department of Genetic Epidemiology in Psychiatry, Central Institute of Mental Health, Medical Faculty Mannheim, Heidelberg University, Mannheim, Germany
| | - Eva Z Reininghaus
- Department of Psychiatry and Psychotherapeutic Medicine, Medical University of Graz, Graz, Austria
| | - Alessio Squassina
- Section of Neuroscience and Clinical Pharmacology, Department of Biomedical Sciences, University of Cagliari, Italy
| | - Lucia Valmaggia
- South London and Maudsley NHS Foundation Trust, London, UK; Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK; Department of Psychiatry, KU Leuven, Belgium
| | - Lars Vedel Kessing
- Copenhagen Affective disorder Research Center (CADIC), Psychiatric Center Copenhagen, Denmark; Department of clinical Medicine, University of Copenhagen, Denmark
| | - Eduard Vieta
- Hospital Clinic, Institute of Neuroscience, University of Barcelona, IDIBAPS, CIBERSAM, Barcelona, Catalonia, Spain
| | - Christoph U Correll
- The Zucker Hillside Hospital, Department of Psychiatry, Northwell Health, Glen Oaks, NY, USA; Department of Psychiatry and Molecular Medicine, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, USA; Center for Psychiatric Neuroscience; The Feinstein Institutes for Medical Research, Manhasset, NY, USA; Department of Child and Adolescent Psychiatry, Charité Universitätsmedizin, Berlin, Germany
| | - Celso Arango
- Department of Child and Adolescent Psychiatry, Institute of Psychiatry and Mental Health, Hospital General Universitario Gregorio Marañón, Gregorio Marañón; Health Research Institute (IiGSM), School of Medicine, Universidad Complutense de Madrid; Biomedical Research Center for Mental Health (CIBERSAM), Madrid, Spain
| | - Ole A Andreassen
- NORMENT, Institute of Clinical Medicine, University of Oslo and Division of Mental Health and Addiction, Oslo University Hospital, Oslo, Norway
| | | |
Collapse
|
42
|
Moving towards vertically integrated artificial intelligence development. NPJ Digit Med 2022; 5:143. [PMID: 36104535 PMCID: PMC9474277 DOI: 10.1038/s41746-022-00690-x] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2022] [Accepted: 08/31/2022] [Indexed: 11/08/2022] Open
Abstract
AbstractSubstantial interest and investment in clinical artificial intelligence (AI) research has not resulted in widespread translation to deployed AI solutions. Current attention has focused on bias and explainability in AI algorithm development, external validity and model generalisability, and lack of equity and representation in existing data. While of great importance, these considerations also reflect a model-centric approach seen in published clinical AI research, which focuses on optimising architecture and performance of an AI model on best available datasets. However, even robustly built models using state-of-the-art algorithms may fail once tested in realistic environments due to unpredictability of real-world conditions, out-of-dataset scenarios, characteristics of deployment infrastructure, and lack of added value to clinical workflows relative to cost and potential clinical risks. In this perspective, we define a vertically integrated approach to AI development that incorporates early, cross-disciplinary, consideration of impact evaluation, data lifecycles, and AI production, and explore its implementation in two contrasting AI development pipelines: a scalable “AI factory” (Mayo Clinic, Rochester, United States), and an end-to-end cervical cancer screening platform for resource poor settings (Paps AI, Mbarara, Uganda). We provide practical recommendations for implementers, and discuss future challenges and novel approaches (including a decentralised federated architecture being developed in the NHS (AI4VBH, London, UK)). Growth in global clinical AI research continues unabated, and introduction of vertically integrated teams and development practices can increase the translational potential of future clinical AI projects.
Collapse
|
43
|
Afrose S, Song W, Nemeroff CB, Lu C, Yao D. Subpopulation-specific machine learning prognosis for underrepresented patients with double prioritized bias correction. COMMUNICATIONS MEDICINE 2022; 2:111. [PMID: 36059892 PMCID: PMC9436942 DOI: 10.1038/s43856-022-00165-w] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2021] [Accepted: 07/27/2022] [Indexed: 11/09/2022] Open
Abstract
Abstract
Background
Many clinical datasets are intrinsically imbalanced, dominated by overwhelming majority groups. Off-the-shelf machine learning models that optimize the prognosis of majority patient types (e.g., healthy class) may cause substantial errors on the minority prediction class (e.g., disease class) and demographic subgroups (e.g., Black or young patients). In the typical one-machine-learning-model-fits-all paradigm, racial and age disparities are likely to exist, but unreported. In addition, some widely used whole-population metrics give misleading results.
Methods
We design a double prioritized (DP) bias correction technique to mitigate representational biases in machine learning-based prognosis. Our method trains customized machine learning models for specific ethnicity or age groups, a substantial departure from the one-model-predicts-all convention. We compare with other sampling and reweighting techniques in mortality and cancer survivability prediction tasks.
Results
We first provide empirical evidence showing various prediction deficiencies in a typical machine learning setting without bias correction. For example, missed death cases are 3.14 times higher than missed survival cases for mortality prediction. Then, we show DP consistently boosts the minority class recall for underrepresented groups, by up to 38.0%. DP also reduces relative disparities across race and age groups, e.g., up to 88.0% better than the 8 existing sampling solutions in terms of the relative disparity of minority class recall. Cross-race and cross-age-group evaluation also suggests the need for subpopulation-specific machine learning models.
Conclusions
Biases exist in the widely accepted one-machine-learning-model-fits-all-population approach. We invent a bias correction method that produces specialized machine learning prognostication models for underrepresented racial and age groups. This technique may reduce potentially life-threatening prediction mistakes for minority populations.
Collapse
|
44
|
Plana D, Shung DL, Grimshaw AA, Saraf A, Sung JJY, Kann BH. Randomized Clinical Trials of Machine Learning Interventions in Health Care: A Systematic Review. JAMA Netw Open 2022; 5:e2233946. [PMID: 36173632 PMCID: PMC9523495 DOI: 10.1001/jamanetworkopen.2022.33946] [Citation(s) in RCA: 47] [Impact Index Per Article: 23.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/16/2022] Open
Abstract
IMPORTANCE Despite the potential of machine learning to improve multiple aspects of patient care, barriers to clinical adoption remain. Randomized clinical trials (RCTs) are often a prerequisite to large-scale clinical adoption of an intervention, and important questions remain regarding how machine learning interventions are being incorporated into clinical trials in health care. OBJECTIVE To systematically examine the design, reporting standards, risk of bias, and inclusivity of RCTs for medical machine learning interventions. EVIDENCE REVIEW In this systematic review, the Cochrane Library, Google Scholar, Ovid Embase, Ovid MEDLINE, PubMed, Scopus, and Web of Science Core Collection online databases were searched and citation chasing was done to find relevant articles published from the inception of each database to October 15, 2021. Search terms for machine learning, clinical decision-making, and RCTs were used. Exclusion criteria included implementation of a non-RCT design, absence of original data, and evaluation of nonclinical interventions. Data were extracted from published articles. Trial characteristics, including primary intervention, demographics, adherence to the CONSORT-AI reporting guideline, and Cochrane risk of bias were analyzed. FINDINGS Literature search yielded 19 737 articles, of which 41 RCTs involved a median of 294 participants (range, 17-2488 participants). A total of 16 RCTS (39%) were published in 2021, 21 (51%) were conducted at single sites, and 15 (37%) involved endoscopy. No trials adhered to all CONSORT-AI standards. Common reasons for nonadherence were not assessing poor-quality or unavailable input data (38 trials [93%]), not analyzing performance errors (38 [93%]), and not including a statement regarding code or algorithm availability (37 [90%]). Overall risk of bias was high in 7 trials (17%). Of 11 trials (27%) that reported race and ethnicity data, the median proportion of participants from underrepresented minority groups was 21% (range, 0%-51%). CONCLUSIONS AND RELEVANCE This systematic review found that despite the large number of medical machine learning-based algorithms in development, few RCTs for these technologies have been conducted. Among published RCTs, there was high variability in adherence to reporting standards and risk of bias and a lack of participants from underrepresented minority groups. These findings merit attention and should be considered in future RCT design and reporting.
Collapse
Affiliation(s)
| | - Dennis L Shung
- Department of Medicine, Yale University, New Haven, Connecticut
| | - Alyssa A Grimshaw
- Harvey Cushing/John Hay Whitney Medical Library, Yale University, New Haven, Connecticut
| | - Anurag Saraf
- Department of Radiation Oncology, Massachusetts General Hospital, Boston, Massachusetts
| | - Joseph J Y Sung
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore
| | - Benjamin H Kann
- Artificial Intelligence in Medicine Program, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts
| |
Collapse
|
45
|
Antúnez-Muiños P, Vicente-Palacios V, Pérez-Sánchez P, Sampedro-Gómez J, Sánchez-Puente A, Dorado-Díaz PI, Nombela-Franco L, Salinas P, Gutiérrez-García H, Amat-Santos I, Peral V, Morcuende A, Asmarats L, Freixa X, Regueiro A, Caneiro-Queija B, Estevez-Loureiro R, Rodés-Cabau J, Sánchez PL, Cruz-González I. Predictive Power for Thrombus Detection after Atrial Appendage Closure: Machine Learning vs. Classical Methods. J Pers Med 2022; 12:jpm12091413. [PMID: 36143197 PMCID: PMC9503612 DOI: 10.3390/jpm12091413] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Revised: 08/12/2022] [Accepted: 08/21/2022] [Indexed: 11/16/2022] Open
Abstract
Device-related thrombus (DRT) after left atrial appendage (LAA) closure is infrequent but correlates with an increased risk of thromboembolism. Therefore, the search for DRT predictors is a topic of interest. In the literature, multivariable methods have been used achieving non-consistent results, and to the best of our knowledge, machine learning techniques have not been used yet for thrombus detection after LAA occlusion. Our aim is to compare both methodologies with respect to predictive power and the search for predictors of DRT. To this end, a multicenter study including 1150 patients who underwent LAA closure was analyzed. Two lines of experiments were performed: with and without resampling. Multivariate and machine learning methodologies were applied to both lines. Predictive power and the extracted predictors for all experiments were gathered. ROC curves of 0.5446 and 0.7974 were obtained for multivariate analysis and machine learning without resampling, respectively. However, the resampling experiment showed no significant difference between them (0.52 vs. 0.53 ROC AUC). A difference between the predictors selected was observed, with the multivariable methodology being more stable. These results question the validity of predictors reported in previous studies and demonstrate their disparity. Furthermore, none of the techniques analyzed is superior to the other for these data.
Collapse
Affiliation(s)
- Pablo Antúnez-Muiños
- CIBERCV, University Hospital of Salamanca, 37007 Salamanca, Spain
- Biomedical Research Institute of Salamanca (IBSAL), 37007 Salamanca, Spain
- Correspondence: ; Tel.: +34-923291100
| | | | - Pablo Pérez-Sánchez
- CIBERCV, University Hospital of Salamanca, 37007 Salamanca, Spain
- Biomedical Research Institute of Salamanca (IBSAL), 37007 Salamanca, Spain
| | | | | | - Pedro Ignacio Dorado-Díaz
- CIBERCV, University Hospital of Salamanca, 37007 Salamanca, Spain
- Biomedical Research Institute of Salamanca (IBSAL), 37007 Salamanca, Spain
| | - Luis Nombela-Franco
- Instituto Cardiovascular, Hospital Clínico San Carlos, IdISSC, 28040 Madrid, Spain
| | - Pablo Salinas
- Instituto Cardiovascular, Hospital Clínico San Carlos, IdISSC, 28040 Madrid, Spain
| | - Hipólito Gutiérrez-García
- CIBERCV, Instituto de Ciencias del Corazón (ICICOR), Hospital Clínico Universitario de Valladolid, 47003 Valladolid, Spain
| | - Ignacio Amat-Santos
- CIBERCV, Instituto de Ciencias del Corazón (ICICOR), Hospital Clínico Universitario de Valladolid, 47003 Valladolid, Spain
| | - Vicente Peral
- Department of Cardiology, Health Research Institute of the Balearic Islands (IdISBa), Hospital Universitari Son Espases, 07120 Palma, Spain
| | - Antonio Morcuende
- Department of Cardiology, Health Research Institute of the Balearic Islands (IdISBa), Hospital Universitari Son Espases, 07120 Palma, Spain
| | - Lluis Asmarats
- Quebec Heart and Kung Institute, Laval University, Quebec City, QC G1V 0A6, Canada
| | - Xavier Freixa
- Institut Clínic Cardiovascular, Hospital Clínic, Institut d’Investigacions Biomèdiques August Pi I Sunyer (IDIBAPS), 08036 Barcelona, Spain
| | - Ander Regueiro
- Institut Clínic Cardiovascular, Hospital Clínic, Institut d’Investigacions Biomèdiques August Pi I Sunyer (IDIBAPS), 08036 Barcelona, Spain
| | | | | | - Josep Rodés-Cabau
- Quebec Heart and Kung Institute, Laval University, Quebec City, QC G1V 0A6, Canada
| | - Pedro Luis Sánchez
- CIBERCV, University Hospital of Salamanca, 37007 Salamanca, Spain
- Biomedical Research Institute of Salamanca (IBSAL), 37007 Salamanca, Spain
| | - Ignacio Cruz-González
- CIBERCV, University Hospital of Salamanca, 37007 Salamanca, Spain
- Biomedical Research Institute of Salamanca (IBSAL), 37007 Salamanca, Spain
| |
Collapse
|
46
|
Noori A, Magdamo C, Liu X, Tyagi T, Li Z, Kondepudi A, Alabsi H, Rudmann E, Wilcox D, Brenner L, Robbins GK, Moura L, Zafar S, Benson NM, Hsu J, R Dickson J, Serrano-Pozo A, Hyman BT, Blacker D, Westover MB, Mukerji SS, Das S. Development and Evaluation of a Natural Language Processing Annotation Tool to Facilitate Phenotyping of Cognitive Status in Electronic Health Records: Diagnostic Study. J Med Internet Res 2022; 24:e40384. [PMID: 36040790 PMCID: PMC9472045 DOI: 10.2196/40384] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2022] [Revised: 07/29/2022] [Accepted: 07/31/2022] [Indexed: 11/13/2022] Open
Abstract
BACKGROUND Electronic health records (EHRs) with large sample sizes and rich information offer great potential for dementia research, but current methods of phenotyping cognitive status are not scalable. OBJECTIVE The aim of this study was to evaluate whether natural language processing (NLP)-powered semiautomated annotation can improve the speed and interrater reliability of chart reviews for phenotyping cognitive status. METHODS In this diagnostic study, we developed and evaluated a semiautomated NLP-powered annotation tool (NAT) to facilitate phenotyping of cognitive status. Clinical experts adjudicated the cognitive status of 627 patients at Mass General Brigham (MGB) health care, using NAT or traditional chart reviews. Patient charts contained EHR data from two data sets: (1) records from January 1, 2017, to December 31, 2018, for 100 Medicare beneficiaries from the MGB Accountable Care Organization and (2) records from 2 years prior to COVID-19 diagnosis to the date of COVID-19 diagnosis for 527 MGB patients. All EHR data from the relevant period were extracted; diagnosis codes, medications, and laboratory test values were processed and summarized; clinical notes were processed through an NLP pipeline; and a web tool was developed to present an integrated view of all data. Cognitive status was rated as cognitively normal, cognitively impaired, or undetermined. Assessment time and interrater agreement of NAT compared to manual chart reviews for cognitive status phenotyping was evaluated. RESULTS NAT adjudication provided higher interrater agreement (Cohen κ=0.89 vs κ=0.80) and significant speed up (time difference mean 1.4, SD 1.3 minutes; P<.001; ratio median 2.2, min-max 0.4-20) over manual chart reviews. There was moderate agreement with manual chart reviews (Cohen κ=0.67). In the cases that exhibited disagreement with manual chart reviews, NAT adjudication was able to produce assessments that had broader clinical consensus due to its integrated view of highlighted relevant information and semiautomated NLP features. CONCLUSIONS NAT adjudication improves the speed and interrater reliability for phenotyping cognitive status compared to manual chart reviews. This study underscores the potential of an NLP-based clinically adjudicated method to build large-scale dementia research cohorts from EHRs.
Collapse
Affiliation(s)
- Ayush Noori
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
| | - Colin Magdamo
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
| | - Xiao Liu
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
| | - Tanish Tyagi
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
| | - Zhaozhi Li
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
| | - Akhil Kondepudi
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
| | - Haitham Alabsi
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
- Harvard Medical School, Boston, MA, United States
| | - Emily Rudmann
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
- Vaccine and Immunotherapy Center, Division of Infectious Disease, Boston, MA, United States
| | - Douglas Wilcox
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
- Harvard Medical School, Boston, MA, United States
| | - Laura Brenner
- Harvard Medical School, Boston, MA, United States
- Division of Pulmonary and Critical Care Medicine, Massachusetts General Hospital, Boston, MA, United States
| | - Gregory K Robbins
- Harvard Medical School, Boston, MA, United States
- Division of Infectious Diseases, Massachusetts General Hospital, Boston, MA, United States
| | - Lidia Moura
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
- Harvard Medical School, Boston, MA, United States
| | - Sahar Zafar
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
- Harvard Medical School, Boston, MA, United States
| | - Nicole M Benson
- Harvard Medical School, Boston, MA, United States
- Mongan Institute, Massachusetts General Hospital, Boston, MA, United States
- McLean Hospital, Belmont, MA, United States
| | - John Hsu
- Harvard Medical School, Boston, MA, United States
- Mongan Institute, Massachusetts General Hospital, Boston, MA, United States
| | - John R Dickson
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
- Harvard Medical School, Boston, MA, United States
| | - Alberto Serrano-Pozo
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
- Harvard Medical School, Boston, MA, United States
| | - Bradley T Hyman
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
- Harvard Medical School, Boston, MA, United States
| | - Deborah Blacker
- Harvard Medical School, Boston, MA, United States
- Department of Psychiatry, Massachusetts General Hospital, Boston, MA, United States
| | - M Brandon Westover
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
- Harvard Medical School, Boston, MA, United States
| | - Shibani S Mukerji
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
- Harvard Medical School, Boston, MA, United States
- Division of Infectious Diseases, Massachusetts General Hospital, Boston, MA, United States
| | - Sudeshna Das
- Department of Neurology, Massachusetts General Hospital, Boston, MA, United States
- Harvard Medical School, Boston, MA, United States
| |
Collapse
|
47
|
Khene ZE, Bigot P, Doumerc N, Ouzaid I, Boissier R, Nouhaud FX, Albiges L, Bernhard JC, Ingels A, Borchiellini D, Kammerer-Jacquet S, Rioux-Leclercq N, Roupret M, Acosta O, De Crevoisier R, Bensalah K. Application of Machine Learning Models to Predict Recurrence After Surgical Resection of Nonmetastatic Renal Cell Carcinoma. Eur Urol Oncol 2022:S2588-9311(22)00137-7. [PMID: 35987730 DOI: 10.1016/j.euo.2022.07.007] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2022] [Revised: 05/28/2022] [Accepted: 07/21/2022] [Indexed: 11/04/2022]
Abstract
BACKGROUND Predictive tools can be useful for adapting surveillance or including patients in adjuvant trials after surgical resection of nonmetastatic renal cell carcinoma (RCC). Current models have been built using traditional statistical modelling and prespecified variables, which limits their performance. OBJECTIVE To investigate the performance of machine learning (ML) framework to predict recurrence after RCC surgery and compare them with current validated models. DESIGN, SETTING, AND PARTICIPANTS In this observational study, we derived and tested several ML-based models (Random Survival Forests [RSF], Survival Support Vector Machines [S-SVM], and Extreme Gradient Boosting [XG boost]) to predict recurrence of patients who underwent radical or partial nephrectomy for a nonmetastatic RCC, between 2013 and 2020, at 21 French medical centres. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS The primary end point was disease-free survival. Model discrimination was assessed using the concordance index (c-index), and calibration was assessed using the Brier score. ML models were compared with four conventional prognostic models, using decision curve analysis (DCA). RESULTS AND LIMITATIONS A total of 4067 patients were included in this study (3253 in the development cohort and 814 in the validation cohort). Most tumours (69%) were clear cell RCC, 40% were of high grade (nuclear International Society of Urological Pathology grade 3 or 4), and 24% had necrosis. Of the patients, 4% had nodal involvement. After a median follow-up of 57 mo (interquartile range 29-76), 523 (13%) patients recurred. ML models obtained higher c-index values than conventional models. The RSF yielded the highest c-index values (0.794), followed by S-SVM (c-index 0.784) and XG boost (c-index 0.782). In addition, all models showed good calibration with low integrated Brier scores (all integrated brier scores <0.1). However, we found calibration drift over time for all models, albeit with a smaller magnitude for ML models. Finally, DCA showed an incremental net benefit from all ML models compared with conventional models currently used in practice. CONCLUSIONS Applying ML approaches to predict recurrence following surgical resection of RCC resulted in better prediction than that of current validated models available in clinical practice. However, there is still room for improvement, which may come from the integration of novel biological and/or imaging biomarkers. PATIENT SUMMARY We found that artificial intelligence algorithms could better predict the risk of recurrence after surgery for a localised kidney cancer. These algorithms may help better select patients who will benefit from medical treatment after surgery.
Collapse
Affiliation(s)
- Zine-Eddine Khene
- Department of Urology, University of Rennes 1, Rennes, France; LTSI, Inserm U1099, Université de Rennes 1, Rennes, France
| | - Pierre Bigot
- Department of Urology, University of Angers, Angers, France
| | - Nicolas Doumerc
- Department of Urology, University of Toulouse, Toulouse, France
| | - Idir Ouzaid
- Department of Urology, Bichat Claude Bernard Hospital, Paris, France
| | - Romain Boissier
- Department of Urology, Aix-Marseille University, Marseille, France
| | | | - Laurence Albiges
- Department of Medical Oncology, Gustave Roussy, Villejuif, France
| | | | | | | | | | | | - Morgan Roupret
- Department of Urology, La Pitie Salpétrière Hospital, Paris, France
| | - Oscar Acosta
- LTSI, Inserm U1099, Université de Rennes 1, Rennes, France
| | - Renaud De Crevoisier
- LTSI, Inserm U1099, Université de Rennes 1, Rennes, France; Department of Medical Oncology, Centre Eugene Marquis, Rennes, France
| | - Karim Bensalah
- Department of Urology, University of Rennes 1, Rennes, France; LTSI, Inserm U1099, Université de Rennes 1, Rennes, France.
| | | |
Collapse
|
48
|
e-Health in Vascular Diseases: Integrating Digital Innovation in Everyday Clinical Practice. J Clin Med 2022; 11:jcm11164757. [PMID: 36012995 PMCID: PMC9410488 DOI: 10.3390/jcm11164757] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2022] [Accepted: 08/12/2022] [Indexed: 12/11/2022] Open
|
49
|
van Smeden M, Heinze G, Van Calster B, Asselbergs FW, Vardas PE, Bruining N, de Jaegere P, Moore JH, Denaxas S, Boulesteix AL, Moons KGM. Critical appraisal of artificial intelligence-based prediction models for cardiovascular disease. Eur Heart J 2022; 43:2921-2930. [PMID: 35639667 PMCID: PMC9443991 DOI: 10.1093/eurheartj/ehac238] [Citation(s) in RCA: 41] [Impact Index Per Article: 20.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Revised: 03/29/2022] [Accepted: 04/26/2022] [Indexed: 11/12/2022] Open
Abstract
The medical field has seen a rapid increase in the development of artificial intelligence (AI)-based prediction models. With the introduction of such AI-based prediction model tools and software in cardiovascular patient care, the cardiovascular researcher and healthcare professional are challenged to understand the opportunities as well as the limitations of the AI-based predictions. In this article, we present 12 critical questions for cardiovascular health professionals to ask when confronted with an AI-based prediction model. We aim to support medical professionals to distinguish the AI-based prediction models that can add value to patient care from the AI that does not.
Collapse
Affiliation(s)
- Maarten van Smeden
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Universiteitsweg 100, 3584 CG Utrecht, The Netherlands
| | - Georg Heinze
- Section for Clinical Biometrics, Center for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna, Vienna, Austria
| | - Ben Van Calster
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium.,EPI Centre, KU Leuven, Leuven, Belgium.,Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, The Netherlands
| | - Folkert W Asselbergs
- Department of Cardiology, Division Heart and Lungs, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.,Institute of Cardiovascular Science, Faculty of Population Health Sciences, University College London, London, UK.,Health Data Research UK and Institute of Health Informatics, University College London, London, UK
| | - Panos E Vardas
- Department of Cardiology, Heraklion University Hospital, Heraklion, Greece.,Heart Sector, Hygeia Hospitals Group, Athens, Greece
| | - Nico Bruining
- Department of Cardiology, Erasmus MC , Thorax Center, Rotterdam, The Netherlands
| | - Peter de Jaegere
- Department of Cardiology, Erasmus MC, Thorax Center, Rotterdam, The Netherlands
| | - Jason H Moore
- Department of Computational Biomedicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Spiros Denaxas
- Health Data Research UK and Institute of Health Informatics, University College London, London, UK.,The Alan Turing Institute, London, UK
| | - Anne Laure Boulesteix
- Institute for Medical Information Processing, Biometry and Epidemiology, LMU Munich, Germany
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Universiteitsweg 100, 3584 CG Utrecht, The Netherlands
| |
Collapse
|
50
|
van Smeden M. A Very Short List of Common Pitfalls in Research Design, Data Analysis, and Reporting. PRIMER (LEAWOOD, KAN.) 2022; 6:26. [PMID: 36119906 PMCID: PMC9477699 DOI: 10.22454/primer.2022.511416] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Affiliation(s)
- Maarten van Smeden
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| |
Collapse
|